iCloud Sync

[Jaysen]

But as usual I’m going off topic. Saturday ramblings and all that.

Sometimes I think you need to take up drinking with VicK as a hobby.I love to see the "ramblings" that would result from an all night bender with a hairy-arsed-welder or four.

[AmberV]

I find it comforting that you only sometimes think that would be a good idea.

To Cutty Sark.

[Rayz]

The point I was trying to illustrate is that Dropbox is way more than just a sync service now with the Slack et al integration.

If that is your problem with Dropbox, have you looked at the huge list of bloated things that iCloud handles? It sprawls as badly as iTunes.

iCloud is a complicated and over-engineered monster. I alluded to it above, but just because Apple hides this thing inside the operating system does not make it a slick, lightweight and cool. What that really means is that you have no choice in whether it is installed or not, and they can sprawl it deep into stuff that sync should have no business messing with in my opinion—and I state that as an ethical concern as well as a technical one. Just look at it this way: you have to give Dropbox permission to look at your photos. Did you give iCloud permission to do anything? Apple doesn’t ask permission, they just upload your address book to the Internet like that’s cool to do, whereas in most cases we’d call that spyware.

But Dropbox integrates with a development toolkit, so it is clunky and bloated—abandon ship. Sure, sure.

Now the 3-device account limit on their freebie service, yeah I get the grumbles there (despite the whole Gift Horse Mouth bit), and like I say I get the preference aspect, that is fine. But technical arguments for Dropbox being worse than iCloud just make my eyebrows rise.

Clearly, Apple is really good at selling stuff, while Dropbox does not care so much about that kind of thing. Now to a person like myself, that is actually not a point in Apple’s case; Bose is good at selling things too.

I don't think there is much to be gained by calling one system 'bloated' by comparing with another, when the two systems have different goals. iCloud is threaded throughout the operating system because Apple's customer base would probably get quite annoyed if they had keep track of syncing activities across multiple devices. Having it tied throughout the OS means that they can implement stuff like storage optimisation and have all apps code that meet their coding guidelines work without too much of a headache. Of secondary concern to Apple is how the design would look to folk of a more technical bent, because people who are concerned about such things are not really their core user base. What they care about is that it starts up and it works, without having to manually sync.

The problem to my mind, and the thing to pay attention to with this kind of technology, is not the affirmations of when it works correctly, but what happens when it does not. Because what happens when it fails is what we have to prepare for even when it seems to work fine for years. We post advisories about not using Google Drive even though the failure rates are a fraction of the people who use it (many of whom have used it for years). But because those fail states happen often enough, and because their condition is so catastrophic, we warn people away from the service entirely.

I'm not familiar with how Google drive works, but these cloud systems are no different than local hard disks; they will fail, so always make sure you have a backup, but that is a different issue. When they work, then your average user expects them to work without regular intervention. For me, iCloud works without me worrying about what's going on in the background, or how Apple has threaded iCloud throughout the operating system. It's not an everyday problem because, for the most part, it works everyday. It even works when I share the editing of a document with someone else.

A tiny change to a Scapple file (editing one note), resulted in a 45 minute wait for the file to sync. In the meantime if I had not realised that and assumed iCloud were as snappy as other services, I would have conflicted the file

Forty-five minutes is definitely excessive, but I'm not sure it's the norm, and if it's not the norm then I certainly wouldn't want Apple to implement manual syncing across the board to get around it. I would rather they found the problem and fixed it.

This is the same sort of thing you would have to do with iCloud if you messed up—but as I pointed out above, the experience is a whole lot more opaque and difficult to recover from.

How difficult depends on what you're doing. Apple's own apps will alert you to a conflict and ask you to review and select a file. Not sure what other apps do, because, as I said, I don't seem to see these problems.

The opaqueness of the file system is one of those design decisions you either agree or disagree with. Keeping the APIs as abstract as possible means they can do things like move millions of users to a new file system without any of them noticing.

For myself, this kind of catastrophic failure over a rather common misuse scenario is something I have a “One Strike You’re Out” policy on. When I saw iCloud destroy data, that was it. I had no interest in that system and I never will. I have never seen any other sync technology do anything like that, and every other sync tool I have tested has extensive safety nets built in, like archival folders, versioning and trash-buffer systems. Maybe iCloud is better now than it was when I tested it—but like I say, at this point I do not care and I never will, because any system published with the flaws I saw has said enough about itself.

That's fair enough, but I think we may be digressing. You had problems, and threw the whole thing out the window. I have never had a problem, and have backups for when I do, so all I see is a lot of apps that sync seamlessly, and one that doesn't.

Dropbox:

You open Scrivener and it alerts you to the fact that Dropbox has created duplicate conflict files.
They are imported into the binder into one convenient location and cross-referenced to the items they are associated with if possible, making it easy to compare the two copies side by side.

Good example, but let's see it from another perspective:

Dropbox:

You make sure that Scrivener is closed down on your Mac. You leave the house.

You open Scrivener on the bus, and wonder if you closed it down on your Mac. You put it out of your mind because you have the beginnings of a best seller you want to crack on with while you're on the bus.

You wait anything between 15 seconds and 2 minutes for Scrivener to get ready.

You crack on.

You get home before you even start Scrivener, Dropbox tells you you have 18 file conflicts.

You open Scrivener, and it says you have somewhat less.

You spend fifteen minutes comparing files which are all the same, except one where you changed 'their' to 'there'.

I understand your hatred of iCloud, and I will no doubt hate it too if I run into a problem, but I'm really just talking about a problem I'm seeing regularly, rather than one that hasn't happened and for which I'm sort of prepared for.
You're talking about the unreliability of iCloud.
I'm talking about the less than seamless workings of Scrivener syncing which I don't see in other apps, for whatever reason.

One thing worth noting is that rewriting Scrivener from the ground up is a euphemism for gutting pretty much all of the things that make it a unique tool for writers, with regards to large-scale research and text storage. We’re talking turning Scrivener into Yet Another Single File Text Editor, with all of the limitations that come with storing 200,000 words in a single file (and nothing much beyond those words, no research).

The other thing worth noting, and I see this misconception all over the place, is that unless Scrivener were gutted, then if we’re speaking of a hypothetical universe wherin Apple finally adds safe and simple support for bulk folder+file sync, using the iOS protocols—nothing would likely change about the procedure.

Well, I was simply quoting your chief programmer from earlier on in this thread

Yes, I realised a while back that moving to iCloud would not solve the problem, which kind of brings me back to my original message (before I got myself sidetracked), I don't think there is any point in blaming Apple or thinking they will change. I could be wrong but I suspect that they prefer that developers didn't delve into low-level data structures any more; they'd rather they stick to the standard APIs and guidelines so that their apps won't don't get orphaned when the make sweepingly inconvenient moves – like changing the packaging structure altogether.

And if it's fair to criticise Apple for burning iCloud into the OS, then it's also fair to criticise building an app in such a way as to tie a key operation, and possibly its future, to one particular vendor.
I personally don't make that criticism of Apple or L&L because I don't really know the full technical details of why those decisions were made, so I assume that they were made to serve the majority of the users.

[Silverdragon]

@AmberV, thank you for articulating exactly why I loathe iCloud and merely get annoyed at Dropbox.

And---continuing to buy Apple hardware upgrades *is* a subscription fee, of sorts… not going to add to it by subscribing to more iCloud space or Apple Music.

*exits grumbling to go to her writing group meeting*

[Jaysen]

I find it comforting that you only sometimes think that would be a good idea.

To Cutty Sark.

It has to be "only sometimes". Otherwise there would be too many of us free to roam the forum sans a properly sober caretaker to clean up the monkey mess we'd make with our off-topic-ing-via-KIWCS-posts.

I bet that's going to show up in the forum search log quite a bit over the next 24hr.

[lunk]

This is really interesting conversation.
But most interesting is the reaction from some of the posters who just refuse to see the world as it is.

L&L built a Mac app.
When iOS came into life some time after that, some users said: ”It’d be nice with an iOS companion app”
So L&L made one but also said: ”Here it is, but it has limitations because of technical reasons”
The users then said: ”No, we won’t accept any limitations. If you don’t fix it WE WILL USE SOME OTHER APP!!”
When L&L responded: ”Sorry, it is what it is” the users started screaming because they had no intention of abandoning the Mac app. Like spoiled children they were used to getting their wishes granted if they screamed long enough. They screamed and screamed, but the world is what it is, so eventually they had to accept reality and either accept the limitations or go for another app.

But which they chose, that’s another story.

[scshrugged]

In case someone lands here and gets the impression that syncing live projects is the only way to achieve an efficient Scrivener workflow, be aware that KB made other methods possible. Besides Scrivener manuals and knowledge base articles, some additional resources:

There's AirDrop–– https://support.apple.com/en-us/HT203106

iTunes File Sharing–– https://support.apple.com/en-us/HT201301


Apple Files app––
(Apple general info) https://support.apple.com/en-us/HT206481

(Post by Scrivener developer, KB) viewtopic.php?p=239174#p239174


ZIP transfer––
(Post by L&L's, AmberV, regarding the file manager, FileApp) viewtopic.php?p=271859#p271859

(Post regarding Apple's Shortcut app; includes info links) viewtopic.php?p=287729#p287729


I have synced Scrivener Mac to Mac using iCloud Drive (didn't use Desktop and Documents or Optimize features) and it worked flawlessly, other than an operator error(s). I did it *just because*. It was never something I planned on sticking with, especially for serious work. Why add complexity––more points of potential failure? I guess syncing (whichever service) live projects does save a little time, but after the habits to use time-stamped ZIP files are formed, it can't be much. ZIPs can be mailed and/or stored in any number of places for online or offline purposes.

I'm not trying to convert anyone (well, maybe a little bit). The main point––if Scrivener's design makes you productive, it's important to remember that you're not limited to syncing live projects, whether from Mac to Mac, Mac to iOS, iOS to Mac or iOS to iOS.

[Silverdragon]

The other thing worth noting, and I see this misconception all over the place, is that unless Scrivener were gutted, then if we’re speaking of a hypothetical universe wherin Apple finally adds safe and simple support for bulk folder+file sync, using the iOS protocols—nothing would likely change about the procedure.

Folks, in case you missed it—and it was buried in AmberV's long post—this means that if iCloud sync ever comes to iOS Scrivener, it will still not happen "seamlessly" in the background. Instead, we will get an iCloud-branded dialog box that shows syncing progress rather than a Dropbox-branded dialog box. This would be equally true for Box, Google Drive (Heaven forbid), or any other syncing service.

What AmberV said was that syncing will not happen in the background on iOS unless Scrivener is gutted. All platforms. Any sync service. End of story. If this is what you want, you are extremely unlikely to get it.

If what you want instead is a syncing alternative, any alternative, to Dropbox sync for iOS Scrivener, that may happen. It's not feasible given the present state of APIs of other syncing services (including iCloud) but it's conceivable that those APIs (including iCloud's) might change to make it feasible. Until that happy day, our choices will be limited to Dropbox sync or some form of file transfer. @ScShrugged's excellent post above lists file transfer solutions that have been devised to date by Scrivener developers and users.

[kewms]

I don't think there is much to be gained by calling one system 'bloated' by comparing with another, when the two systems have different goals. iCloud is threaded throughout the operating system because Apple's customer base would probably get quite annoyed if they had keep track of syncing activities across multiple devices. Having it tied throughout the OS means that they can implement stuff like storage optimisation and have all apps code that meet their coding guidelines work without too much of a headache.

Spoken like someone who has never tried to help a user whose data was "optimized" into the Bit Bucket, never to be seen again.

Katherine

[kewms]

Good example, but let's see it from another perspective:

Dropbox:

You make sure that Scrivener is closed down on your Mac. You leave the house.

You open Scrivener on the bus, and wonder if you closed it down on your Mac. You put it out of your mind because you have the beginnings of a best seller you want to crack on with while you're on the bus.

You wait anything between 15 seconds and 2 minutes for Scrivener to get ready.

You crack on.

You get home before you even start Scrivener, Dropbox tells you you have 18 file conflicts.

You open Scrivener, and it says you have somewhat less.

You spend fifteen minutes comparing files which are all the same, except one where you changed 'their' to 'there'.

Mac Scrivener has an auto-close option for exactly this reason. I use and recommend it to anyone who uses Scrivener across multiple devices.

If you routinely have dozens of file conflicts, there is probably an issue with your synchronization practices. If you used another system, either you would have the same reported conflicts or you would lose data. I would recommend finding the underlying cause rather than blaming the conduit.

Katherine

[devinganger]

Hmm, on that score though, Apple does not have the best track record for sticking to what they started, or responsibly migrating users to their replacements when they move on.

Nor does Microsoft. OS vendors are no more stable than third-party vendors when it comes to peripheral services.

[JoRo]

I suspect that in time it won’t really matter whether Dropbox is the best solution for the current build of Scrivener or not: what will really determine the future is if Scrivener can maintain its marketshare while only offering Dropbox as a syncing solution. And perhaps even that won’t matter, as Keith, being the sole developer, clearly has the freedom to act as he wants, irrespective of any other concerns.

The underlying issue here is that Apple has not only moved the goalposts in recent years, but it has also ripped up the pitch, torn down the stadium, and changed the rules of the game. All of the pro-Dropbox points I have read in recent threads are all to do with how things were, not with how things are now or clearly will be in the future.

Apple has changed both the design language of apps and the ease-of-use that makes people choose Apple devices across OS platforms. For many users – and that’s a growing number of users in my experience – that means apps that mirror Apple’s design ethos and which use iCloud. Against that backdrop, Scrivener is increasingly seen as being behind the curve.

I know a young writer who has just signed a significant book deal. She used to use Scrivener, but she has dropped it in favour of Ulysses, as its design across OSes is far more in line with Apple’s core ethos and because iCloud works so well.

And this really is the issue: there is a group of long-term users on the forum who love Scrivener for what it was and is, and there is a growing swell of newer (or potential) users who just want to work the Apple way, without having to get a handle on Scrivener 3’s design ethic or larking around with Dropbox.

I think Amber is right: that means gutting Scrivener and rebuilding it again. And I suspect that is exactly what many Scrivener users want now and certainly what new users will expect in the future – we can’t forever live in the noughties.

I have no direct investment in this debate as I only use iCloud for syncing, and I no longer maintain a Dropbox account. If Scrivener’s design across iOS and macOS was consistent, and if syncing was offered through iCloud, I would purchase Scrivener for iOS. As it is, I stick with Scrivener on macOS and Writer for occasional needs on iOS, using external folder sync if necessary.

The founded-on-quicksand reasons presented so far for persisting with Dropbox really don’t have any merit as far as I am concerned, irrespective of how passionately they are presented: the tide is moving away from what we have now, and Scrivener is going to get left in its wake if it doesn’t jump on its surfboard soon and ride the wave. Personally, that’s fine with me as I am happy to choose Scrivener – as it presently is – above other design and syncing preferences. But I am definitely in a dwindling minority in terms of the writers around me, which wasn’t the case a few years ago. From that fact, everything else flows, blowing Dropbox out of the water, even if some people still love it and want to defend it.

My only reason for supporting any move to gut and rebuild Scrivener is that I think it will give Scrivener a longer lifespan, and that really would be ideal for me as I plan to write for at least another 50 years. Selfish, yes.

Just to be transparent: I have also dumped my fax machine and my DVD player. Radical that I am.

Slàinte mhòr.

[devinganger]

I think Amber is right: that means gutting Scrivener and rebuilding it again. And I suspect that is exactly what many Scrivener users want now and certainly what new users will expect in the future

What many of you do not realize is that 1) this is a significant amount of work, a lot more than even the rewrite between v2 and v3, and that 2) some of the core features of Scrivener -- what makes Scrivener what it is, what makes it different and appealing -- may not survive such a theoretical rewrite.

I will also point out that all of your prophesying about what the future of technology holds is just a WAG, just like the rest of us. I've worked in IT for over 25 years now. I can't tell you how many times I've heard predictions of paradigm shifts...but somehow, the old paradigm managed to stick around side-by-side. They're useful and they're not as easily given up. Apple may indeed one day abandon the richness of MacOS -- but my personal bet is that won't happen for a lot longer than you think.

[lunk]

One thing worth noting is that rewriting Scrivener from the ground up is a euphemism for gutting pretty much all of the things that make it a unique tool for writers, with regards to large-scale research and text storage. We’re talking turning Scrivener into Yet Another Single File Text Editor, with all of the limitations that come with storing 200,000 words in a single file (and nothing much beyond those words, no research).

I think Amber is right: that means gutting Scrivener and rebuilding it again. And I suspect that is exactly what many Scrivener users want now and certainly what new users will expect in the future

If that is really what Mac users want, there is a simple solution: use Pages.
Pages is a single-file-text-editor, it syncs seamlessly with iCloud, it allows simultaneous editing by co-writers, and on top of that it is free.

But I think most Scrivener users want the ability to have research material within the project and to be able to split the master piece into smaller chunks, have keywords, labels, etc, i.e all those things that make Scrivener what it is.

Tanstafl! You can’t have it all, so you have to make a choice: Scrivener with its functions or Pages with its syncing etc.

[Brammy]

Mac Scrivener has an auto-close option for exactly this reason. I use and recommend it to anyone who uses Scrivener across multiple devices.

If you routinely have dozens of file conflicts, there is probably an issue with your synchronization practices. If you used another system, either you would have the same reported conflicts or you would lose data. I would recommend finding the underlying cause rather than blaming the conduit.

Katherine

Yeah. I have a bunch of automation set up for this. I use Scriv's auto quit, I also have Quitter set up to make sure it quits, and I have a Keyboard Maestro rule to quit Scriv when it detects a sleep event.