Clarification of iCloud concerns

da
darinb
Posts: 33
Joined: Mon Sep 25, 2017 11:16 pm
Platform: Mac

Wed Oct 04, 2017 11:48 pm Post

New user here.

I've read about the problems of using Scrivener with iCloud but I'm still not clear--many (all?) of the articles are concerned with iPad use.

On my Macs I have the Desktop set to be stored in iCloud (and thus available directly on all my machines). Is this a safe way to use Scrivener--if not does that mean I have to turn off this otherwise very useful feature of the OS...or?

Just need to make 100% sure I'm not heading for a disaster before I start to use the program more intensively... :)

--Darin

Sp
SpringfieldMH
Posts: 587
Joined: Fri Feb 03, 2012 1:18 am
Platform: Mac, Win + iOS

Thu Oct 05, 2017 2:14 am Post

As best I understand from following the forums for the past few years, if you store\sync live Scrivener projects on iCloud, you are heading for disaster. DropBox is the only such service cloud storage/sync service that L&L formally recommends, for technical reasons.

For background, do a search of the forums (upper right hand corner) using "iCloud"
http://literatureandlatte.com/forum/sea ... rds=icloud
and also see these two knowledgebase articles
https://scrivener.tenderapp.com/help/kb ... c-services
https://scrivener.tenderapp.com/help/kb ... g-with-ios

Be aware that even with DropBox, one must exercise some care (discussed in first of above knowledgebase articles).

On the other hand, compressed (.zip) versions of projects should be safe on most all services. But of course one must first decompress (unzip) such off such services to local drive before being able to work on them in Scrivener, and later recompress (zip) them from local drive back to such services.

Hope that is more help than hurt.
I'm a Scrivener enthusiast and experienced in PC/network support... but I am not a Scrivener expert. And I'm still editing on my first book. So take any opinion or advice I offer with a large grain of salt.

da
darinb
Posts: 33
Joined: Mon Sep 25, 2017 11:16 pm
Platform: Mac

Thu Oct 05, 2017 6:26 pm Post

Ouch. That hurts. :)

Since *every* document on my laptop is auto-synced with iCloud, turning it off the satisfy the requirement for one program is probably not workable.

My god, don't tell me I'm forced to go with a subscription-model product--I detest the subscription model concept!

Any clues if version 3 will have a solution for this issue?

--Darin

User avatar
kewms
Posts: 4777
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Thu Oct 05, 2017 6:32 pm Post

darinb wrote:Any clues if version 3 will have a solution for this issue?

--Darin


It will not. This is fundamentally an Apple issue, not a Scrivener issue. While Scrivener 3 includes some changes to the project format, the basic structure -- lots of small files of multiple different file types -- remains the same, and Apple's difficulty in managing it is the root cause.

Katherine
Scrivener Support Team

da
darinb
Posts: 33
Joined: Mon Sep 25, 2017 11:16 pm
Platform: Mac

Thu Oct 05, 2017 6:40 pm Post

kewms wrote:It will not.
Katherine


Ouch, again. Sounds like Scrivener is inherently unusable on my system. Now what do I do? :(

--Darin

User avatar
kewms
Posts: 4777
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Thu Oct 05, 2017 6:50 pm Post

iCloud *Drive* is a reasonably safe location for Scrivener projects. (Subject to the caveats that apply to any synchronization service.) That's what I'd do. (Well, actually, I use Dropbox. But if you refuse to use it for whatever reason.)

Katherine
Scrivener Support Team

da
darinb
Posts: 33
Joined: Mon Sep 25, 2017 11:16 pm
Platform: Mac

Fri Oct 06, 2017 12:20 am Post

kewms wrote:iCloud *Drive* is a reasonably safe location for Scrivener projects. (Subject to the caveats that apply to any synchronization service.) That's what I'd do.

Katherine


Oh no, I'm confused again. I'm not entirely sure of the distinction between "syncing" and "iCloud Drive." All my Desktop files are automatically shares across my computers. There is a folder on my iCloud Drive labeled "Desktop and the files are all in there. I don't manually do anything--files are automatically put there (and updated there).

is this the dangerous thing or the reasonable safe thing?

--Darin

User avatar
kewms
Posts: 4777
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Fri Oct 06, 2017 1:12 am Post

Apple's naming conventions can be somewhat confusing...

iCloud is a collection of Apple services for sharing data between iCloud-enabled applications. These services are not supported by Scrivener, and therefore iCloud syncing will not work for sharing projects between, say, Mac OS and iOS.

iCloud Drive is a virtual hard drive that works similarly to Dropbox. Therefore, in theory, it is safe for Scrivener projects, provided that you only use it for backups or for sharing between Mac OS systems. (Subject to the caveats that apply to any synchronization system.)

HOWEVER, the Documents and Desktop feature of Mac OS is *extremely* risky if the amount of data you want to store is larger than your iCloud subscription allows. Which it will be for many people, as the "free" allocation is only 5 GB. There are all sorts of reports of alarming behavior if you either overrun your space allocation or try to turn the feature off. I do not use this feature for my personal data and do not recommend it for naive users.

Katherine
Scrivener Support Team

Lu
LuckyJack
Posts: 40
Joined: Thu Aug 24, 2017 2:39 am
Platform: Mac + iOS

Fri Oct 06, 2017 3:24 am Post

kewms wrote:iCloud *Drive* is a reasonably safe location for Scrivener projects. (Subject to the caveats that apply to any synchronization service.) That's what I'd do. (Well, actually, I use Dropbox. But if you refuse to use it for whatever reason.)

Katherine


I stored Scrivener projects in iCloud Drive without any problems, but decided I would sync a USB drive to a normal folder based on the warnings.

But I never understood why iCloud Drive wouldn't work, and I didn't see any troubles. it syncs files. Lots of them, if need be.

I'd like to see clarification on this, too. If iCloud is safe, assuming you wait for sync to complete on power up and shutdown of any device. What's the downside?

User avatar
kewms
Posts: 4777
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Fri Oct 06, 2017 4:30 am Post

LuckyJack wrote:I stored Scrivener projects in iCloud Drive without any problems, but decided I would sync a USB drive to a normal folder based on the warnings.

But I never understood why iCloud Drive wouldn't work, and I didn't see any troubles. it syncs files. Lots of them, if need be.

I'd like to see clarification on this, too. If iCloud is safe, assuming you wait for sync to complete on power up and shutdown of any device. What's the downside?


Note that this entire conversation only applies to Mac OS systems. iOS Scrivener has to deal with additional complexity because of the way files are managed on iOS devices, which is why we're only able to support Dropbox synchronization. Mac OS has a real file system with a library of standard tools for file operations.

In theory, all of these services should work: Dropbox, iCloud Drive, Google Drive, and several others all do pretty much the same thing. They all create a special area on your disk that is synchronized with their server.

In practice, it's important that (1) the software give a clear indication that your changes have in fact been synchronized, (2) the software upload all changes promptly in the order in which they were made, and (3) the software NOT get cute with 'cleanup' features like duplicate removal.

(1) Is a matter of user interface design. Should be easy, but somehow seems to elude some services. iCloud Drive is not clear about this, which is one reason why we don't trust it enough to use internally.

(2) Seems to be the underlying cause of Google Drive's poor performance. Scrivener saves very frequently, and it can cause *enormous* problems if the 9:15 PM version of a project has only partially uploaded when the 9:20 PM save occurs. (This is also why it's a good idea to pause synchronization if you know your internet connection is poor.)

(3) Some services try to save you disk space by removing duplicate files. That's a terrible idea if, say, you really did want to have three different copies of your author bio, one for each installment of a trilogy. Or if you have five or six backup copies in which most (but not all!) of the files are the same. This behavior seems to be more common in tools that started out as backup services and morphed into synchronization services as the market changed.

Katherine
Scrivener Support Team

br
brookter
Posts: 1819
Joined: Wed Mar 18, 2009 12:22 pm
Platform: Mac

Fri Oct 06, 2017 7:22 am Post

darinb wrote:
Oh no, I'm confused again. I'm not entirely sure of the distinction between "syncing" and "iCloud Drive." All my Desktop files are automatically shares across my computers. There is a folder on my iCloud Drive labeled "Desktop and the files are all in there. I don't manually do anything--files are automatically put there (and updated there).

is this the dangerous thing or the reasonable safe thing?

--Darin


Darin,

It's not really clear to me either. BUT in any case there is a very easy solution/workaround which should avoid any problems arising. [1]

1. Create a new Folder (let's call it Scrivener Documents) which isn't in your iCloud Drive. The best place to do this is in your Home folder (in Finder Go > Home or cmd-shift-H). Drag the new folder into the side bar for easy access.

2. Copy your existing Scrivener projects into Scrivener Documents, and make sure you create all new Scrivener projects there.

3. In Scrivener Preferences go to Backup and make sure you are creating regular automatic backups INSIDE the iCloud Drive and that they are Zipped / Compressed. (Best is to up the number to 25 as well...)

This means you'll be able to work on the current files safely outside iCloud's automatic syncing and that the backups will be synced into iCloud for safety (all flavours of iCloud work fine with zipped files). The distinction between various types of iCloud won't matter then.

It also means that the current files are only available on the one Mac — if you want to do want to share them easily on more than one Mac, then most people use Dropbox. My Current Work folder is on Dropbox on both my laptop and desktop and I work on Scrivener projects there -- all I need to do is remember to close the project on one machine before I open it on the other. Backups are zipped and are in iCloud Drive. (Make sure your Dropbox folder is also outside the iCloud Drive and directly in Home -- I think that's the default anyway.)


[1] Note to those more knowledgable than I -- if there's anything wrong with the above please let Darin know!

User avatar
KB
Site Admin
Posts: 20690
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Fri Oct 06, 2017 10:09 am Post

Hi,

Katherine has given lots of good answers here, so I'm probably not adding much, but:

Basically, it's perfectly safe to store your files on iCloud Drive, nearly as much as it is on Dropbox. However, with any cloud service, you just have to be careful to make sure that files are fully synced between machines before you use them. So, if you edit a project on computer A, you need to be sure that the changes have uploaded to the cloud service from machine A and also downloaded to computer B before you use the project on computer B. And you need to have the project closed on the computer on which you are not using it to avoid conflicts.

As long as you follow these rules, you will be safe on iCloud Drive or Dropbox.

The downside of iCloud is that it is not as transparent about when it syncs files as Dropbox. Dropbox has an indicator right in the menu bar showing when it is syncing. As soon as you make a change to a file in Dropbox, it starts uploading. And you can see the sync indicator on your other computer to check that things are downloading. For iCloud, you have to find the file in the Finder, because the upload/download progress is shown there, not in an easy-to-see place such as the binder.

Also, Dropbox is much better at making conflicted file copies than iCloud Drive. If you do accidentally work on a project that hasn't finished syncing, Dropbox will make file conflict copies, and Scrivener can retrieve them. You'll then get a message in Scrivener telling you that unexpected files were found and you'll have to decide how to handle them - an annoyance, to be sure, but at least you can recover your work. On iCloud, conflicts are just lost, so you *could* lose work if you aren't careful about following the rules of ensuring that everything is synced before you open and work on a project.

In most apps, there are no real complications, because most apps have a file format that consists of a single file per document/project. But a Scrivener project is really a folder containing many files - thus allowing the user to bring in research files, and allowing Scrivener to run fast even for massive projects, because it only has to load into memory the files it needs for the current session (whereas single-file apps have to load the entire file into memory). So Scrivener is essentially a hybrid between a shoebox app (a UI for a repository of many files, such as Photos) and a document-based app (an app in which each window operates on a single file, such as Pages). iCloud wasn't built with this sort of app in mind, because most apps are one or the other rather than a hybrid. This is why you have to be a little more careful in Scrivener.

Ultimately, though, I would just say: if things are working for you on iCloud Drive, just carry on. Just be sure to have Scrivener's automatic backups feature turned on, so that if anything does ever go wrong, you can easily restore from a backup.

All the best,
Keith
"You can't waltz in here, use my toaster, and start spouting universal truths without qualification."

Lu
LuckyJack
Posts: 40
Joined: Thu Aug 24, 2017 2:39 am
Platform: Mac + iOS

Fri Oct 06, 2017 3:48 pm Post

Keith, thanks for that great information regarding iCloud Drive issues.

There are a couple of ways to see iCloud sync. In the Finder, a little clock face will show sync progress in the sidebar (view -> show side bar, or command-option-s) if you have a shortcut to iCloud there. The clock face will appear beside the iCloud shortcut.

Individual things waiting for upload or download will appear in the lists of files with a little cloud symbol.

For more detail, turn on the status bar with view -> show status bar (or command-/). That will show you the available space on iCloud if it's synced, and if it's in the process of syncing, you'll see how much is waiting to transfer.

I think I'll move back to iCloud for my writing - but I'll keep backups turned on in Scrivener to a local, non-shared directory and I'll make manual backups. Time Machine, too, with my rotation of three backup drives.

Belts, suspenders, and hard liquor. No cutting corners with disaster recovery and backups.

User avatar
KB
Site Admin
Posts: 20690
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Fri Oct 06, 2017 3:57 pm Post

Thanks for that extra info, LuckyJack!
"You can't waltz in here, use my toaster, and start spouting universal truths without qualification."

da
darinb
Posts: 33
Joined: Mon Sep 25, 2017 11:16 pm
Platform: Mac

Fri Oct 06, 2017 11:24 pm Post

This has been a very helpful discussion. Thanks to everyone.

Regarding iCloud Drive, it seems to summary is:

1) iCloud is fine as long as you have enough disk space (not a problem for me, I have the 2 TB plan).
2) iCloud is fine as long as you don't quickly shut the computer before it has time to upload/update files. You can monitor this progress (which seems very quick with a good connection).
3) iCloud is fine as long as you don't keep a copy of the Binder open on one machine and then start using another.

I'd like to explore that third option a bit and then wrap this up by learning what the worst case scenarios are.

It is, of course, common to leave apps open, often for months on end, on current computers. For large writing projects I can easily imagine the document (Binder in this case) might also stay open semi-permanently. What exactly will happen if I leave the app open on my laptop, close the laptop (after iCloud Drive update) and then open that file on my Desktop? And then make changes there (letting it update to iCloud Drive) before going back to the already open Binder on my laptop? In short, must I close the Binder after every use (a habit very much contrary to my habits with other programs)?

Also, what about situations without good (or any) internet connections on one side or the other in that scenario, say when traveling?

For all of these situations (assuming I have auto-backup turned on), what is the worst case scenario? Do I lose actual text that I've written or the research files I've gathered? Or "just" the Binder and all the work I put into organizing the writing project?

I don't want to over-emphasize the worst case scenario aspect. After all, back in the typewriter days writers ran a huge risk in losing original drafts on paper. Even in recent times, before Dropbox and iCloud, it was all too easy to lose files, accidentally deleting them or having a hard drive fail. I suspect that whatever worst-case scenarios are outlined will, by comparison, present a fraction of the risk we routinely endured just a short time ago.

But, that said, what are the worst-case scenarios with iCloud Drive and how might one attempt to mitigate them?

--Darin