Scrivener seems to be struggling with "Downloading file list"

is
istara
Posts: 66
Joined: Sat Nov 04, 2006 7:07 pm

Fri Dec 15, 2017 11:17 am Post

I've noticed that recently, Scrivener's DropBox sync has become incredibly slow. I haven't increased the number of projects and files in my DropBox folder (in fact I even decreased them to try and improve things). I have been using Scrivener iOS since release, and have NOT encountered this issue before. I believe it may have started with the update (v1.1.5) a week ago.

I also tried a clean install. This resulted in Scrivener having to download several thousand files - which of course took a long while - but it did it flawlessly.

However, now when I update/change a single file on desktop, Scrivener is taking forever to update it on iOS. Its initial check is pretty quick, eg a couple of seconds, and I can see the single project file is the only one with a "needing sync" icon.

But then, when I go to sync, Scrivener literally takes minutes on "Downloading file list". I timed it, and it took 2 minutes and 20 seconds. It then took just a few seconds to update 8 files (all I had changed in the Project was one single word, just for a test).

The same happened when I tried to sync back up. I added one more word to one Project, and it again took 2 minutes and 20 seconds for "Downloading file list". The actual changes then synced up in a few seconds (like 5 seconds).

My suspicion is that for some reason, Scrivener iOS is checking every single file in every project rather than the single file it has identified as needing updating.

The project file I have been testing this on is not large. It's about 2.2mb (it contains a cover image) and is ~50,000 words spread across around 60 scenes.

ub
ub
Posts: 49
Joined: Thu Jan 10, 2008 1:35 pm
Platform: Mac + iOS

Sat Dec 16, 2017 12:40 pm Post

I've seen the same behavior. Downloading file list is markedly slower now.

Online
User avatar
lunk
Posts: 4063
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Sat Dec 16, 2017 12:49 pm Post

I don’t see it.

How many projects do you have in your sync folder?
I am a user, writing non-fiction and science, using:
* Mac Scrivener 3 on a Macbook 12”, MacBook Pro 13”, and iMac 27”, all running Mojave.
* iOS Scrivener 1 on an iPhone 11 Pro, iPad Air 9.7”, and iPad Pro 12.9”, all running the latest iOS

is
istara
Posts: 66
Joined: Sat Nov 04, 2006 7:07 pm

Sat Dec 16, 2017 12:53 pm Post

lunk wrote:I don’t see it.

How many projects do you have in your sync folder?


40. And I have always had about the same number in there.

Online
User avatar
lunk
Posts: 4063
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Sat Dec 16, 2017 12:57 pm Post

I have 9.

Why not simply reduce the number of projects you actively sync? It would free space in your iDevice’s memory as well. On your Mac, create a subfolder in your Scrivener projects folder and move your avtive projects there. On the iDevice, change the dropbox settings in Scrivener so it looks in the subfolder, and see if there is a change in sync time.
I am a user, writing non-fiction and science, using:
* Mac Scrivener 3 on a Macbook 12”, MacBook Pro 13”, and iMac 27”, all running Mojave.
* iOS Scrivener 1 on an iPhone 11 Pro, iPad Air 9.7”, and iPad Pro 12.9”, all running the latest iOS

is
istara
Posts: 66
Joined: Sat Nov 04, 2006 7:07 pm

Sat Dec 16, 2017 1:02 pm Post

lunk wrote:I have 9.

Why not simply reduce the number of projects you actively sync? It would free space in your iDevice’s memory as well. On your Mac, create a subfolder in your Scrivener projects folder and move your avtive projects there. On the iDevice, change the dropbox settings in Scrivener so it looks in the subfolder, and see if there is a change in sync time.


Because I actually need all these projects! I'm about to go on a 10 day road trip with just my iPad, and it will be the greatest hassle to keep logging into Dropbox and manually moving stuff and redownloading entire Project files back and forth just to speed up sync.

I'm struggling to understand how this has just recently become a problem, when it wasn't before. All the tests I have done indicate the software is almost certainly checking through all the files of all the projects when it doesn't need to (or didn't before).

I do appreciate that you are trying to help, but it seems clear to me that something has happened with this update that has changed the way sync behaves.

Online
User avatar
lunk
Posts: 4063
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Sat Dec 16, 2017 1:15 pm Post

I have the latest updates of both OS X and iOS Scrivener, and I don’t see what you see. It is still instantaneous.

Do you have anything else than the Scrivner projects in the sync folder? Any other files or subfolders?
I am a user, writing non-fiction and science, using:
* Mac Scrivener 3 on a Macbook 12”, MacBook Pro 13”, and iMac 27”, all running Mojave.
* iOS Scrivener 1 on an iPhone 11 Pro, iPad Air 9.7”, and iPad Pro 12.9”, all running the latest iOS

is
istara
Posts: 66
Joined: Sat Nov 04, 2006 7:07 pm

Sat Dec 16, 2017 1:47 pm Post

lunk wrote:I have the latest updates of both OS X and iOS Scrivener, and I don’t see what you see. It is still instantaneous.

Do you have anything else than the Scrivner projects in the sync folder? Any other files or subfolders?


Nothing. And as detailed in the other thread, I have tried completely reinstalling and copying across the Project files one by one.

Have you attempted to replicate this with a large number of Project files? Eg 30 Project files of considerable size each?

The initial check for what needs updating (eg on launch) is still near instantaneous. It's only when you go to actually sync that the delay occurs.

si
sidderke
Posts: 186
Joined: Thu Dec 03, 2015 11:34 pm
Platform: Mac

Sat Dec 16, 2017 3:31 pm Post

I have the same problem.
It's really not just a bit slower, but it used to be instantaneous, now it takes literally 5 minutes.
This is only since I updated my projects to Scrivener 3.0

I contacted L&L support about it. They said Scrivener 3.0 doubles the amount of files to be indexed, and that's why it takes Dropbox longer to deliver the list with files.
Now, I did tell them that I did not experience a double amount of timing waiting, because it went from seconds to about 5 minutes, which is *a lot* more then double the amount.

They kind of told me they have seen that behaviour before, with people who have a lot of projects in their Dropbox OR very big ones. I don't have that much projects, but they are very big.
To be honest, I'm a bit dissapointed as well, they didn't offer a solution except having smaller projects on Dropbox. This wasn't an issue on Scrivener 2 projects, and I thought the 3.0 was gonna make everything work faster instead of slower. I don't mind if I had to wait 30 seconds instead of 15 seconds, but going from seconds to 5 minutes or more, just waiting, really is noticeable. Especially if you are waiting for just a quick sync of a couple of meta data texts, nothing big.

The syncing of the files itself go just as quick as before, it's just that Downloading File List.
I do hope the team can figure out what is causing this. I'm a bit dissapointed they brush it off as normal, while the old version didn't have this, and it's a LOT more then double the amount of time waiting.

mi
mijalce
Posts: 17
Joined: Fri Aug 14, 2015 9:55 pm
Platform: Mac
Contact:

Sat Dec 16, 2017 8:41 pm Post

For me Scrivener is very important tool in my writing workflow. Thus, I wanted to see do I make something wrong and to improve it. There is a suggestion that Scrivener syncs big projects (too many files or too many GB) more slowly and it was implicitly suggested that we need to reorganise our projects to meet this requirement. On the other hand, I and others were telling that the same projects with Scrivener 2 was synced extremely fast. Thus I made the following experiment.
I have 18 projects in my dropbox folder. The biggest is 32 MB, all the projects in total are 137MB. These projects as Scrivener 3 version when synced with Scrivener on iOS take around 3 and half minutes. 99% of time in status "Downloading file list". Then, on MacOS I made no changes to the projects just exported them in Scrivener 2 version and added them in new folder in Dropbox. I downloaded Scrivener 2.9 and opened each Scrivener 2 version project. I made no changes. Waited to sync everything on Dropbox. Then I went in iOS and unliked Dropbox and deleted local files. Then I relinked and pointed to the folder where Scrivener 2 version projects are. After the initial sync which lasts long I initiated a new sync and it took approximately 12 seconds to finish the sync. Then, on macOS I opened my largest Scrivener 2 version project and added one file. I synced it with Dropbox and then went on iOS where I was informed that there is a change on macOS I accepted the sync and sync was made in 12 seconds, majority of time is actual sync of files.
Then I unlinked dropbox in iOS and stopped syncing dropbox on macOS. I opened Scrivener 2 version projects in Scrivener 3 and upgraded them. I deleted the back up that it is created during the conversion, activated Dropbox and synced the Scrivener 3 version projects on Dropbox. Then went in iOS and synced the Scrivener 3 version projects. After the successful initial sync I manually triggered a new sync and again it took almost 3 minutes with 99% of the time "Downloading file list". I used the same projects all the time, no additional projects, no additional files.
The result of this experiment is that there is some change in Scrivener 3 that does not handle well the communication to Dropbox about what has changed in Scrivener 3 on macOS. Thus, I think that the problem is not the number or size of the files and we should focus the effort on identifying what is changed in Scrivener 3 that makes sync much longer. I can make additional tests that can help you in that process.

si
sidderke
Posts: 186
Joined: Thu Dec 03, 2015 11:34 pm
Platform: Mac

Sat Dec 16, 2017 8:50 pm Post

Thanks for doing the tests, Mijalce!

So in your case it was a difference between 12 seconds and 3 minutes. It confirms that it's not just the doubling of indexing files that takes double the amount of time. My experience was the same, I went from a lot bigger projects (8GB big in the 'worst' case) but the syncing that I was doing was often a small amount of small files, like 5 meta data files because of changing a label, or just adding a sentence in 3 documents. This also went from seconds to 5 minutes in my case.

is
istara
Posts: 66
Joined: Sat Nov 04, 2006 7:07 pm

Sat Dec 16, 2017 10:56 pm Post

sidderke wrote:Thanks for doing the tests, Mijalce!

So in your case it was a difference between 12 seconds and 3 minutes. It confirms that it's not just the doubling of indexing files that takes double the amount of time. My experience was the same, I went from a lot bigger projects (8GB big in the 'worst' case) but the syncing that I was doing was often a small amount of small files, like 5 meta data files because of changing a label, or just adding a sentence in 3 documents. This also went from seconds to 5 minutes in my case.


If you look at my tests, the sole factor that seems to increase the slowness is the number and size of Projects in the folder. Instead of checking and updating the single file identified as changed, it is clearly checking every file in every single Project.

* with 1-5 Project files in the Dropbox folder, syncing a single Project file takes about 5-10 seconds
* with ~25 Project files in the Dropbox folder, syncing a single Project file takes about 30 seconds
* with ~40 Project files in the Dropbox folder, syncing a single Project file takes about 55-60 seconds

Le
Leena
Posts: 16
Joined: Fri Dec 28, 2012 1:58 pm
Platform: Windows

Mon Dec 18, 2017 11:44 am Post

My iPhone won't synch Scrivener with Dropbox at all any more. I believe the last time I managed to do so was almost a month ago. When I click sync, the app announces that all the files are up to date, but in fact does not bring to the phone the files I have written with my Mac and saved on Dropbox. The last four chapters of my latest book, for example.

Leena

Online
User avatar
lunk
Posts: 4063
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Mon Dec 18, 2017 12:46 pm Post

Start the Dropbox app and make sure you are logged in on Dropbox.
I am a user, writing non-fiction and science, using:
* Mac Scrivener 3 on a Macbook 12”, MacBook Pro 13”, and iMac 27”, all running Mojave.
* iOS Scrivener 1 on an iPhone 11 Pro, iPad Air 9.7”, and iPad Pro 12.9”, all running the latest iOS

Le
Leena
Posts: 16
Joined: Fri Dec 28, 2012 1:58 pm
Platform: Windows

Mon Dec 18, 2017 4:06 pm Post

I've opened the Mac desktop Dropbox, the online Dropbox, the Dropbox app on iPhone. Scrivener is linked with Dropbox. Doesn't sync.

Leena