HAVE I LOST EVERYTHING 2 YRS OF WORK ON SCRIVENER??

cf
cfblakeman
Posts: 29
Joined: Fri Mar 13, 2009 5:34 am

Fri Mar 13, 2009 5:39 am Post

Opened Scrivener and it asked me to upgrade to the newest version. I did so, and now every document I was working on is blank. All the files and folders are there and all of them appear to be blank - thousands of pages and 2 years of work on three books.

I'm really hoping there is some simple thing I'm missing here. Please let me know if someone can throw some light on this quick.

And yes, I back everthing up, but before I noticed everything was blank I had already backed up my computer, overwriting the previous backup with these apparent blank files/folders.

Help?

User avatar
KB
Site Admin
Posts: 20901
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Fri Mar 13, 2009 10:10 am Post

First, do not empty your Trash folder.

Second, the update does nothing - nothing to your files.

Third, were you adding work to the Tutorial project? The Tutorial project gets overwritten by a new version with each update to account for new features, and you should not be using the Tutorial project for your own work. If you have inadvertently done this, check this thread:

viewtopic.php?f=2&t=5553#p45462

If this isn't the case, please give more details of what happened. When you update, the only thing affected is the Scrivener application itself. It does not update your files or anything else.

All the best,
Keith

ma
matt
Posts: 1180
Joined: Mon Jul 30, 2007 9:35 am

Fri Mar 13, 2009 11:09 pm Post

I've been holding off until you got some other advice, but...

When this is sorted, you really need to examine your backup system. A system that copies directly over your previous backup is NOT a backup system at all. If something goes wrong during the copy,you lose both backups at once. Always keep at least 2 - your most recent and your previous, and for important files back them up in at least 2 different ways (local hard drive and offsite). Time machine is great for the first, if you use leopard. And look into scrivener's backup to zip option.

-------------

Having said all that. Assuming you didn't work directly into the tutorial. Can you find the actual files on disk in the finder? You can search in spotlight to find them. If you do have the files, there are some steps we can run through to get the data back out if scrivener doesn't open them properly.

Matt

User avatar
KB
Site Admin
Posts: 20901
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Sat Mar 14, 2009 9:32 am Post

Thanks, Matt. In an e-mail conversion we have determined that the op was apparently adding all of his work to the tutorial project. I haven't heard back, but hopefully the information I gave him no how to retrieve his work will have resolved the issue.

And just to reiterate to all users: Do not save work in the tutorial project. Even though Scrivener now asks you to save your tutorial to disk so that these problems shouldn't happen in future, it's still not a great way of starting your project. File > New Project is the place to create your own projects.

Thanks and all the best,
Keith

Ea
EastCoast
Posts: 15
Joined: Thu Jan 17, 2008 5:41 pm
Location: Washington, DC

Sun Mar 15, 2009 7:57 pm Post

I posted a couple of weeks ago under Christopher Adams. I see here that you say the upgrade does nothing to one's files. It certainly did something to mine, and I don't appear to be alone. I upgraded to 1.5 and not long after that my new files disappeared. I am not a computer idiot. I did not erase them. I have searched my hard drive for anything with "scriv." My newest files are simply not there. I searched again today. The files have not magically reappeared. Needless to say, I am not happy. At least two people have bought Scrivener on my recommendation but I certainly couldn't recommend this.

User avatar
KB
Site Admin
Posts: 20901
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Sun Mar 15, 2009 8:44 pm Post

Sorry, I missed your previous post. The thing is, that the update definitely does nothing to your files. I know this for a fact - I programmed it. So in all these cases there must be something else at play that we haven't got to the bottom of yet. In the vast majority of cases, as with the one in this post, it has been caused by a user saving work in the Tutorial project (which one should not do as that will cause lost work). There is simply no way that Scrivener can just wipe projects from your computer during the update process. All that happens is that Sparkle (the external update framework that many programs use) replaces the current version of Scrivener with a the new one. All .scriv files (except the Tutorial one that used to be stored inside the Scrivener applications itself) are not touched in any way.

In your original post, you say that problems first started happening when you noticed that Scrivener - the application - was only 54kb. Given that you had been working in it before (so it couldn't have been the result of a bad update), and that this is just something that happened when you came to your computer, Scrivener itself could not have done this. Scrivener cannot start eating itself - this is just not possible. The only explanation is that something external to Scrivener did this. You also say that the extras wouldn't install and that you had a problem opening the DMG - all of which would suggest a problem on your computer (a permissions problem, perhaps?).

I know you say that you did nothing odd on your computer and that everything else is working fine, but this does not mean that Scrivener is to blame. I understand that it is frustrating to lose work as a user, but it is equally frustrating as a developer, knowing as I do that there is nothing in Scrivener that can wipe projects in this manner.

Are you using any backup or sync software?

All the best,
Keith

Ea
EastCoast
Posts: 15
Joined: Thu Jan 17, 2008 5:41 pm
Location: Washington, DC

Mon Mar 16, 2009 3:15 pm Post

Responding in no particular order: I do use a backup system. It backs up automatically every night but like another poster I didn't notice the files were missing until a couple of backups later. (I alternate backup drives so that I have two days of backups at all times.) It just now occurs to me that the missing files may have been archived and I will check for that when I finish this and hope that's the case. However, the files have disappeared from my hard drive. I did not save them into the tutorial, but if that's a common mistake then there must be something that can be done to ensure that doesn't happen. I have been using the previous version of Scrivener for at least a year with no problems. What happened with me is exactly what Adam Christopher described in his posts of March 2 at 9:03 and 9:55 pm, and it happened only after I upgraded. Everything else on my computer seems to be OK. There are no other missing files that I know of, and I would certainly know of them.

Frankly, at this point I'm not entirely sure exactly what files are missing. Retract that statement. There are three files in question. A version of the first file from the summer of 2008 still exists. There may be other versions of that file, too, but not the latest versions. The second file appears to be intact although the notes don't look identical to the notes I remember. The third file exists because I printed it. If I find it on disk, I'll let you know immediately.

If this is not a problem with the Scrivener update but something I have done inadvertently, then I seem to have lots of company. I'm sure this is as frustrating to you as it is to me and the others who've had the same or a similar problem. But I can't ascribe my problems to a malicious girlfriend or someone fooling around with my computer or any of the other causes various posters have suggested. I'm the only person who's touched it and I know how to use it. I check permissions regularly and run the disk utility or Disk Warrior once or twice a month, and occasionally fsck. The whole thing is bewildering.

User avatar
KB
Site Admin
Posts: 20901
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Mon Mar 16, 2009 4:19 pm Post

It is bewildering and I'm not ruling anything out or accusing users of not knowing what they are doing. I just want to avoid a panic here, as these cases are rare and I just cannot see how Scrivener itself could do anything like this. In Adam Christopher's case, he says that everything got reverted to a state from a year ago. This is quite a precise thing to happen and would involve a program being able to remember the states of files as they were a year ago and to be able to restore them to that state. Scrivener simply does not have that ability, any more than does TextEdit or Pages; it knows nothing of your files until you open them (the list in the Recent Projects menu being stored in the preferences and handled by OS X itself). And then, it only knows of them in their current state. So it is simply impossible for Scrivener to revert a file to an earlier state. The only sort of software that could do this is backup software.

During the update process, all that happens again is for Scrivener itself to get replaced. No files are touched - the update process just does not have that ability.

So this is what I mean that there must be something else going on.

Could you please list all of the software that you use for backup or that you have running in the background for maintenance tasks and so forth? It occurs to me that by gathering such a list it may turn out to be a bad combination - perhaps some software that does not like Scrivener's package format?

Thanks and all the best,
Keith

Si
Sirmausalot
Posts: 15
Joined: Wed Dec 10, 2008 12:39 pm

Mon Mar 16, 2009 4:56 pm Post

I back up important files in various places these days: drop box, thumb drive, e-mail to myself. And as I point out in another post, a hard copy print out is essential every once in a while. Never trust computers (or yourself to use them properly).

That said, I really feel for those who have lost work. I hope it is recovered. I'd also suggest a big warning that tells people to back up to an external device before upgrading (or even does it automatically). anyhow, good luck to all.

User avatar
ptram
Posts: 956
Joined: Sun Mar 25, 2007 3:43 pm
Platform: Mac
Location: Recanati, Italy

Mon Mar 16, 2009 11:56 pm Post

Some times ago I got an empty Scrivener project when zipping, uploading and downloading it between Leopard and my remote ftp server. Maybe Keith can remember my (indeed, not very detailed) report.

My guess, in that case, was that I started uploading the file before Leopard had actually finished zipping. The zip file looked fine in the Finder, but it was probably not yet ready. So, I was uploading an incomplete zip file, and when getting it back it was a damaged project.

I wonder if the same is happening when backing up files: they are copied to a new location before they are completely compressed. Maybe this has something to do with the complex structure of Scrivener project files, and a problem on the compression routine's side to understand it? (Not being a software engineer, I'm just trying with a sci-fi hypothesis).

Paolo

User avatar
Jaysen
Posts: 6294
Joined: Mon Dec 17, 2007 4:00 am
Platform: Mac + Windows
Location: East-Be-Jesus-Nowhere SC, USA

Tue Mar 17, 2009 12:28 am Post

ptram has brought up a good point (and I am not sure KB can do anything about it). Writes are buffered into kernel space to speed up IO functions for apps. Since the buffer is implemented per app file handle a second app will not see the data until the kernel can flush the buffer to disk.

Think of it like this: You are trying to get me to understand a logical argument about why the sky is blue. I think MUCH slower than you talk so you get AmberV to buffer your argument by listening to you and then repeating it at a speed I can understand. This allows you to go on to do something else while she deals with the "slow guy". Now imagine that there are 19 additional people arguing with me, one of me, and AmberV can spontaneously create a clone of herself. The limit of the clone is that a clone can only talk to me and ONE person. Clones can not talk to each other. As you can see the clones will start to get a bit of a backlog pretty quick because I am slow and you are fast.

Now lets say that one person asks me to give back the argument that you just buffered to your AmberV. I start to processes the request for info, but I had to stop your AmberV before she was finished to provide the answer. I will only provide part of the argument because I have only heard part of it.

If we go back to the real world, you and the other people are applications, AmberV is the kernel (specifically the IO interface library), and I am the disk subsystem. App A opens file Z getting file descriptor Za, app B opens file Z getting file handle Zb. A does a write to Za and micro seconds (or seconds depending on IO activity) later B tries to read from Zb. Za is a partial write, but the IO poll moves to Zb and B get a partial read with a EOF. We now have file corruption in B. If B writes to Zb at all we will have Z corrupted on disk.

What can KB do to fix this.
1) Tell everyone to not use real time file sync services. He, AmberV, and a few others have done that.
2) Tell folks not to work on scriv files stored on high latency devices. Think USB sticks, network drives. Done.
3) Implement kernel level file locking. This can be overridden by other programs, so this is really not a solution as much as a "I did all I can, so go back to #1 and #2" and will take a "pile o' work" (that is a technical term, trust me).
4) Lose the package format and go to flat file methods. But then you would have bigger IO actions, and still have the same problem.

All that to say that I believe that scriv is NOT the source of the issue, but that several factors are innocently contributing to this recurring issue. I think the massive IO that scriv accumulates via the short auto save may be a significant factor. Those of you who insist on using real time file sync services should consider raising your auto save to number several times larger than the time it takes to sync your entire scriv file. I can give you some math to calculate that auto save time, but you would need to run several timing apps and I doubt the sync services owners would consent.

Whew.

By the way, if AmberV ever did spontaneously clone herself I am sure she would be a parallel neural with massively shared mem IO. Then this whole problem would go away.

Please don't try to argue with me on the whole file IO buffer thing. The examples here are intended to be understandable, not 100% accurate. I know there are "fibs" by way of simplification. I bet vic-k would understand it enough to explain it to Fluff (or the other way around). And that was the point.
Jaysen

I have a wife and 2 kids that I can only attribute to a wiggle, a giggle, and the realization that she was out of my league so I might as well be happy with her as a friend. 26 years marriage later, I can't imagine life without her. -Me 10/7/09

ImageImage

User avatar
KB
Site Admin
Posts: 20901
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Tue Mar 17, 2009 10:20 am Post

To answer ptram, Scrivener's file structure is no different when it comes to being zipped than any other folder - it doesn't have a particularly complicated structure anyway. And for zipping up projects during backup, Scrivener just uses the zip shell command (which you can use via the Terminal).
All the best,
Keith

Ea
EastCoast
Posts: 15
Joined: Thu Jan 17, 2008 5:41 pm
Location: Washington, DC

Tue Mar 17, 2009 4:08 pm Post

KB wrote:It is bewildering and I'm not ruling anything out or accusing users of not knowing what they are doing. I just want to avoid a panic here, as these cases are rare and I just cannot see how Scrivener itself could do anything like this. In Adam Christopher's case, he says that everything got reverted to a state from a year ago. This is quite a precise thing to happen and would involve a program being able to remember the states of files as they were a year ago and to be able to restore them to that state. Scrivener simply does not have that ability, any more than does TextEdit or Pages; it knows nothing of your files until you open them (the list in the Recent Projects menu being stored in the preferences and handled by OS X itself). And then, it only knows of them in their current state. So it is simply impossible for Scrivener to revert a file to an earlier state. The only sort of software that could do this is backup software.

During the update process, all that happens again is for Scrivener itself to get replaced. No files are touched - the update process just does not have that ability.

So this is what I mean that there must be something else going on.

Could you please list all of the software that you use for backup or that you have running in the background for maintenance tasks and so forth? It occurs to me that by gathering such a list it may turn out to be a bad combination - perhaps some software that does not like Scrivener's package format?

Thanks and all the best,
Keith


Keith,

Thanks for your response. There must be some connection between Adam Christopher and me, since we both reverted to a file that was many months old. I use Carbon Copy Cloner for backing up, and I back up every night, alternating volumes. As far as I am aware, I am not running any maintenance software in the background. The puzzling part of all this is that I had no problem with Scrivener until I updated. I am in the process of moving from this flat-panel iMac running 10.4.11 to a black MacBook running Leopard. I've already installed Scrivener in the new computer, and although I haven't used it for writing, I have used it to open Scivener documents. Everything seems fine. I found I had already moved the second file to the new computer before the problems started, so I have recovered that one and I have faith that I will find the others, too. (I also back up the files I'm working on to a thumb drive; the two missing files were not among them, unfortunaely.)

I accept your statement that Scrivener can't magically retrieve a file from last July and think it's the latest version. Obviously, the problem Adam Christopher and I have is not common or the board would be deluged with messages. This episode has forced me to clean up my files and organize that part of my life a little better so in a sense this has done me a favor, albeit one I would prefer not to have received. Until recently, I've worked on this project too sporadically and saved too many versions in too many folders. I am trashing those that are outdated. I do believe that somewhere I will find the other two files I'm looking for. I bought this computer in July, 2002, and the drive is getting tired. I don't believe that my backup software caused the problem. I am thinking now that Scrivener didn't cause it either, though I have no idea what did.

Thanks again for your concern.
William

User avatar
AmberV_3338G8eG
Posts: 2
Joined: Tue Mar 17, 2009 4:07 pm

Tue Mar 17, 2009 9:57 pm Post

Jaysen wrote:By the way, if AmberV ever did spontaneously clone herself I am sure she would be a parallel neural with massively shared mem IO. Then this whole problem would go away.


We accept nothing less than L1.
.vA
Literature Responsibly Eschews Plausibility