How Does iOS Handle Sync Conflicts?

ca
cammot
Posts: 13
Joined: Sat Aug 02, 2014 7:40 am
Platform: Mac
Location: New Hampshire, USA

Mon Jul 18, 2016 11:25 am Post

KB wrote: The only time you need an internet connection is when you the the "Sync" button, at which point Scrivener uploads any changes made locally and downloads any changes made on the server.


OK - Help me out with this scenario... I appreciate this might be a silly situation, but based on the quoted comment above, it needs to be asked.

Phase 1

All three files are in sync, i.e., iMac, iPad and Dropbox server.

Phase 2

You are out somewhere with iPad and update the file substantially, but do not sync (no connection)

Phase 3

You are back home, and turn on the iMac. Both iMac and Dropbox server are in sync (keep in mind, that you did not sync you iPad with Dropbox server). You proceed to make changes on the iMac and sync with Dropbox server.

===========

With the above 3 phases, you now have a situation, where you are back on the iPad and want to update the Dropbox server with the earlier updated iPad files that was not sync'd.

Question: How does the iPad / Dropbox sync address the version on the Dropbox that was changed prior via the iMac, but not on the iPad before it got changed, too. Please tell me that there's some 'magic', that merges the two distinct versions ! Least a warning flash that you are about to lose something.

cammot

User avatar
lunk
Posts: 3340
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Mon Jul 18, 2016 12:08 pm Post

Working with Dropbox is like working with a remote hard drive. If you have an iMac and an iPad you then have two more hard drives.
Maybe it's easier if you don't use the word "sync"? Think of it as Save... instead. And there is no magic.

From what I understand of how Dropbox works, your scenario would then read like this:

Phase 1 - all three hard drives have the same copy of the file.

Phase 2 - the file on your iPad is changed.

Phase 3 - you change the file on your iMac and the Dropbox app then copies that file to the Dropbox hard drive (in the background, automatically, without you doing anything). The iMac and the Dropbox hard drives both have the same version of the file.

Back on the iPad you tell it to Save ("sync") the iPad version of the file to the Dropbox hard drive, which it does, which means that it overwrites the old version so that the iPad and Dropbox now have identical files on their hard drives. When you go back to the iMac, the Dropbox app will automatically detect that the Dropbox file has changed, and will download it to your iMac, meaning all of your previous changes on the iMac are gone.

Neither machine can possibly know what you think, only what you do. Dropbox is just like an intermediate hard drive, whre files are stored so you can easily copy them from one computer to another. So there is no magic merging.
I am a user, writing non-fiction and science, using:
* Mac Scrivener 3 on a Macbook 12”, MacBook Pro 13”, and iMac 27”, all running the latest MacOS
* iOS Scrivener 1 on an iPhone 8, iPad Air 9.7”, and iPad Pro 12.9”, all running the latest iOS

User avatar
xiamenese
Posts: 3934
Joined: Mon Jan 29, 2007 1:32 am
Platform: Mac
Location: London or Exeter, UK.

Mon Jul 18, 2016 12:32 pm Post

Just don't try to edit a project on either the Mac or iOS device without synchronising it first, or you will get a conflict where the Mac doesn't know the changes on the iOS device, and the iOS device doesn't know about the changes on the Mac. But the new versions implement a way that you can resolve the resulting conflict ... neither machine can do that for you.

Mark
The Scrivenato sometimes known as Mr X.
rMBP 13" (early 2015) 10.14.5, 8GB RAM, 512GB SSID
MBP17" (late 2011) 10.13.6, 8GB RAM, 512GB SSID
2017 iPad, iOS 12.3.1, 128GB, Apple Pencil
Scrivener, Scapple, Nisus Writer Pro, Bookends …

User avatar
KB
Site Admin
Posts: 20701
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Mon Jul 18, 2016 12:54 pm Post

I've split this into its own thread so that it doesn't get in the way of the other user's technical problems.

It's this sort of problem that gave me many months of headaches - and it's exactly the sort of reason I chose the Dropbox API and manual sync in the end.

In this situation, one of two things would happen when you next sync (either on iOS or on the Mac/Windows):

1) You'll receive a message saying that were conflicts were detected and successfully resolved.

Or:

2) You'll receive a message saying that conflicts were detected and some conflicted copies of documents have been placed in a "Conflicts" folder at the bottom of the binder. You would then need to look in this folder and determine whether you want to keep the copies of documents there against the ones that are in the binder, or delete them. (This folder will only contain copies of documents that were edited on both devices in an un-synced state, so there should never be many to go through when this does happen, unless you edit on two devices without syncing for days on end.)

A nice thing about Dropbox is that the desktop client automatically creates conflicted copies of files if it detects that the same file has been edited in two places. Scrivener can then look for such files and add them to the project, finding what document they belong to, and tell the user about the conflicts.

But more importantly - and part of the reason 2.8 is necessary for working with iOS projects - I have updated the way the file format works for iOS. When you edit a project on iOS, it does not directly change the files used by the desktop version. Instead, it "checks out" copies of the files into an extra folder inside the .scriv package. When you open the project on the Mac or Windows, these checked-out files get merged back into the main project. This means that a scenario such as you describe does not result in data loss. Scrivener has access to both copies of the files and can compare them. There are various algorithms involved in this, but basically Scrivener does its best to resolve all conflicts, and won't create conflicted copies of files where one file is unambiguously newer than the other. But it errs on the side of caution: if there is a chance that you have made different edits to a single file, it will create a conflicted file so that you can recover anything you want.

Of course, none of this is ever a problem if you are careful about syncing, which just involves:

1. Always waiting for the Dropbox client on the Mac or Windows to finish updating files before continuing to work on a project.

2. Always hitting "sync" in the iOS version when you know there are pending changes from the desktop version (the iOS version will tell you when it detects changes and prompt you to sync) and when you have made changes and are ready to return to the desktop (projects needing sync will show an indicator so that you know you need to tap sync when you are ready). There is an option in iOS to have it sync automatically whenever you close a project, and if you try to open a project that the iOS version thinks has changes pending from Dropbox, it will warn you and prompt you to sync.

Ultimately, you won't lose any work, but you might have to deal with some conflicts here and there if you don't get into good sync habits. My philosophy is that I would rather have users complain that our sync isn't automatic and "magic" and that we force them to get into good sync habits rather than complain that our sync has lost their data.

All the best,
Keith
"You can't waltz in here, use my toaster, and start spouting universal truths without qualification."

User avatar
yosimiti
Posts: 596
Joined: Sun Nov 09, 2014 5:56 am
Platform: Mac

Mon Jul 18, 2016 5:08 pm Post

Keith, I'll tell you right now...there are gonna a lot of whiners if users have to manually do anything. We're in an age of automatism and people don't like to manually do anything, no matter how much you officiate that it's 'good for you' or that you should develop some kind of discipline. I'm not saying I won't; I'm just giving a heads up that there are gonna be a lot of whiners to this.

In an ideal world, you should be able to work on Scrivener iOS, not have to save in any fancy way, and when you close the app on your device it should automatically save whatever you are working on so that when you open the same file on your desktop you can work on it. The same should go the other way. That is, you work, you close (and not manually save) and open the updated file on iOS. This I hope is the eventual goal...otherwise, the plebians will revolt I say, the plebians will revolt...

Plebians.jpeg
Plebians.jpeg (10.41 KiB) Viewed 2454 times

User avatar
KB
Site Admin
Posts: 20701
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Mon Jul 18, 2016 5:22 pm Post

People will always complain about something. And sadly, we don't live in an ideal world. If you would like to consider all the factors in our sync processes and have a better solution, I'm all ears... But trust me, I have worked *months* on the sync code and considered it from eery angle. Possible if we were Google with whole teams and dedicated servers and APIs, we could make everything seem like "magic". *Sigh.*
"You can't waltz in here, use my toaster, and start spouting universal truths without qualification."

User avatar
lunk
Posts: 3340
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Mon Jul 18, 2016 5:42 pm Post

yosimiti wrote:That is, you work, you close (and not manually save) and open the updated file on iOS. This I hope is the eventual goal...otherwise, the plebians will revolt I say, the plebians will revolt...


Keith knows this a lot better than I do, but from my experience of iOS devices I say that's impossible. You never close iOS apps, you just start using another app, so you leave the Scrivener app for a few seconds or for weeks. How is the app to know? To make it work the way you want it to the Scrivener app would have to save with short intervals, roughly like Mac Scrivener does, but with the difference that the iOS app would have to save to Dropbox all the time, which in turn require a constant WiFi connection.

"But it works for Pages, Drafts, Evernote, etc??"
Yes, sometimes, and only for fairly simple things. iCloud works fairly well for some Apple apps, but only if you work with one file at a time, and only if they are reasonably small. It's the same with Drafts. Often it works, but sometimes even those small snippets of text screw up. Imagine the reaction you would get if the software automatically screw up and destroyed your 150 000 word novel? It's actually much better that people do it themselves.

If all computers – laptops, desktops, iOS, Androids – always had WiFi connection, they could all be interconnected all the time and all machines would know what all the other machines were doing. Then it would easily be solved, except for the major security problems. :P
I am a user, writing non-fiction and science, using:
* Mac Scrivener 3 on a Macbook 12”, MacBook Pro 13”, and iMac 27”, all running the latest MacOS
* iOS Scrivener 1 on an iPhone 8, iPad Air 9.7”, and iPad Pro 12.9”, all running the latest iOS

User avatar
KB
Site Admin
Posts: 20701
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Mon Jul 18, 2016 6:40 pm Post

lunk - yes, basically what we need for flawless sync in file formats such as Scrivener is an internet based on quantum entanglement or spooky action at a distance so that every upload and download is instantaneous, and all computers and devices all connected all the time. And I'm not being facetious - that would be the only way to avoid all problems.

You evaluate the problem well. iCloud does a very good job at syncing "simple" file formats such as TextEdit, Pages, Scapple and such. (By "simple", I really just mean flat - a single file.) But there just is no "magic" solution for syncing a file format such as Scrivener's, where you have a package of interdependent files. Of course I'd *love* it if I could just make it so that all the files synced magically and invisibly and just appeared on your other devices - "just like that" as Paul Daniels used to say. Of course that would be the "eventual goal" - once the technology is there so that it is possible.

The problem lies in the chasm between the expectations raised in users by Apple's "It just works" mantra and the technical realities of syncing a complex file format such as Scrivener's without data loss. It's like the old "fast, good or cheap - pick two" conundrum. I can give users "magic" background sync via Dropbox as long as they don't mind data loss and projects being in unusable states when a connection drops. :)
"You can't waltz in here, use my toaster, and start spouting universal truths without qualification."

si
sidderke
Posts: 134
Joined: Thu Dec 03, 2015 11:34 pm
Platform: Mac

Mon Jul 18, 2016 7:11 pm Post

Don't worry Keith. It sounds like you did a great job thinking about pro and con and you decided that the option that could result in data loss always had to give in to the other option.
Thanks for that.

And having the option of having mobile Scrivener sync automatically on close of the project seems like a nice automated option.

Like in Spider-Man: with great power comes great responsibility. You have the freedom of working on Scrivener projects everywhere now. Having the discipline to think about the sync'ing is a a normal expectation.
And it seems like the app gives you LOTS of safety nets for when you forget once.

User avatar
yosimiti
Posts: 596
Joined: Sun Nov 09, 2014 5:56 am
Platform: Mac

Mon Jul 18, 2016 7:26 pm Post

Please note, I am not on the side of the Plebians; even though I am no programmer, I understand your frustrations, and realize the issue is not simple. But no matter how much I might understand, there will be many who will not. And when it comes to users potentially losing their data over the consequences of your decision, Keith, Rome might fall.

Again, I am no programmer, nor do I pose as one. I am simply the soothsayer, in this scenario, ever so admonishing...beware the ides of march of iOS/MacOS synching....

User avatar
devinganger
Posts: 1413
Joined: Sat Nov 06, 2010 1:55 pm
Platform: Mac, Win + iOS
Location: Monroe, WA 98272 (CN97au)
Contact:

Tue Jul 19, 2016 10:41 am Post

yosimiti wrote:Please note, I am not on the side of the Plebians; even though I am no programmer, I understand your frustrations, and realize the issue is not simple. But no matter how much I might understand, there will be many who will not. And when it comes to users potentially losing their data over the consequences of your decision, Keith, Rome might fall.

Again, I am no programmer, nor do I pose as one. I am simply the soothsayer, in this scenario, ever so admonishing...beware the ides of march of iOS/MacOS synching....


I'm curious -- if you're not arguing a position you yourself don't believe in, why ARE you making such a big fuss about this? If you're trying to be funny, it's not coming off that way on this side of the screen.

Scrivener for iOS is an amazing piece of engineering. The workflow is about as simple (and reliable) as it can get. Keith's done a stellar job of making a very complex problem very usable and safe.
--
Devin L. Ganger, WA7DLG
Not a L&L employee; opinions are those of my cat
Winner "Best in Class", 2018 My First Supervillain Photo Shoot

User avatar
yosimiti
Posts: 596
Joined: Sun Nov 09, 2014 5:56 am
Platform: Mac

Tue Jul 19, 2016 11:21 am Post

I'm curious -- if you're not arguing a position you yourself don't believe in, why ARE you making such a big fuss about this? If you're trying to be funny, it's not coming off that way on this side of the screen.


I'm not trying to be funny. It's a legitimate concern; one that I personally will not be bothered by, but I imagine many will. I see no need to insult.

Scrivener for iOS is an amazing piece of engineering. The workflow is about as simple (and reliable) as it can get. Keith's done a stellar job of making a very complex problem very usable and safe.


I will be the judge of this when the software comes out.

User avatar
rdale
Posts: 1548
Joined: Tue Jul 14, 2015 1:07 pm
Platform: Mac, Win + iOS
Location: St. Louis, MO
Contact:

Tue Jul 19, 2016 12:56 pm Post

yosimiti wrote:
devinganger wrote:Scrivener for iOS is an amazing piece of engineering. The workflow is about as simple (and reliable) as it can get. Keith's done a stellar job of making a very complex problem very usable and safe.

I will be the judge of this when the software comes out.

I'm so glad somebody finally stepped up. I was afraid no one on the entire internet would be the judge. :P
FKA: robertdguthrie
AKA: R Dale Guthrie, Robert, Mr. Obscure, and "Oh, it's you again".

User avatar
yosimiti
Posts: 596
Joined: Sun Nov 09, 2014 5:56 am
Platform: Mac

Tue Jul 19, 2016 1:03 pm Post

Everyone is entitled to my opinion, didn't you know?

User avatar
lunk
Posts: 3340
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Tue Jul 19, 2016 2:55 pm Post

yosimiti wrote:Keith, I'll tell you right now...there are gonna a lot of whiners if users have to manually do anything. We're in an age of automatism and people don't like to manually do anything
(my bold)

Aaahhh... NOW I get it!! :D

I never thought about it like this, but you are absolutely correct. People want spell checkers, software checking grammar, software saving the work when the author forgets to, and wasn't there a link recently about an app that could actually write the body text as well? So you could start the computer in the morning, send the finished novel to a self publishing company by noon, and have the book printed by night fall.

But come to think of it, wouldn't it be even easier to skip the computer and the writing-publishing-printing phase, and simply go down to the nearest book store and buy a ready made book? Or if you don't want to manually do even that, you could have it sent home, or download it so you can listen to it instead of manually flipping the pages.

But thinking a bit more, I think you are dead wrong. People like to do a lot of things manually. And if people can remember to charge their iPad or phone, and to bring it along when they leave the house, surely they can remember to sync their manuscript to Dropbox before they shut down the computer?

Having to remember to do things isn't necessarily a bad thing. ;)
I am a user, writing non-fiction and science, using:
* Mac Scrivener 3 on a Macbook 12”, MacBook Pro 13”, and iMac 27”, all running the latest MacOS
* iOS Scrivener 1 on an iPhone 8, iPad Air 9.7”, and iPad Pro 12.9”, all running the latest iOS