Need help recovering a mangled project (binder index?)

[Dain]

I've had a file causality that has caused some heartache over this year. Short version is that sometime after I put away my NaNo manuscript last year, and this past Feburary when I migrated to a new MBP, several of my projects have suffered from Binder index damage that left those effected projects in shambles. Sadly, since I don't pay DropBox I didn't discover the issue until after I was well outside the lifespan of their versioning tools for free accounts.

I also seem to remember having to load the projects in an updated version of Scrivener on MacOS in order to convert the formats to what iOS Scrivener could use, since I typically would just use my iPad for the bulk of my work.

When I opened these projects, there is a dialog about a sync conflict and I'm presented with a popup asking me to choose which binder information to use. There are three options and there is no clear winner in the preview information. (I know there was nothing updated in this project this year -- this screengrab was taken today).

Screen Shot 2018-10-19 at 1.58.14 PM.png (37.86 KiB) Viewed 1423 times

In each case, I end up with most of the chapter containers having been removed, and a number of files in the recovered folder, most with numberical names (original scene titles missing), worse several versions of the same scene with minor edits.

Screen Shot 2018-10-19 at 2.46.21 PM.png (41.87 KiB) Viewed 1423 times

Thinking I still had the local backups (though no idea where they were or how to access) on the older MBP I decided to switch to a project that would not require hours of recovery editing to fix. Fast forward to the start of August and I had to pick up the pieces from having a car stolen with both my new MacBook and its external TimeMachine drive. DropBox to the rescue as I was fully synced, and almost no words were lost, but it reminded me of the two damaged projects.

When I went to put my old MacBook back into service (the car insurance covered the car but not its contents, and my insurance declined to cover the contents because the car was insured) I replaced the SSDs and copied the contents of my old machine. So its probable I have an intact local backup folder or location. Where would I look for it on MacOS and Windows7 (both are present on that drive)?

Fast forward to today, I was thinking on NaNo and that got me thinking of the two projects I want to rescue, both were NaNo pieces, and both are worthy of being fixed, polished, edited, and finished.

Do I have much hope for finding backups that will make recovery possible? I've taken the step of exporting the numbered scenes by content and plugging them into my programmers IDE and using it's difference engine (a feature I really REALLY wish Scriver offered natively) and found that in most cases the duplicated scenes are incrementally different, like saving the last five versions say. That complicates the recovery process some, and it seems that even then I am missing a dozen scenes or so.

[scshrugged]

It doesn't sound that you've opened and closed any of the projects on the recovered Mac. Before proceeding, make sure that auto-backup on close is disabled (or at least increase the default of five saved to unlimited) in Scrivener Preferences; you can readjust to what you prefer after you get settled. This is to avoid inadvertently over-writing any existing good backups.

Possible ways to find backups:

Use this non-destructive command in Terminal app. It assumes that you used Scrivener's ZIP backup feature:

Code: Select all

find $HOME -name "*.zip" -print

Also, the freeware app EasyFind made available by DEVON Technologies might be of help. You can use it for a wide or narrow scope search using various terms.
http://www.devontechnologies.com/products/freeware.html

In any case if you're fortunate enough to find the backups, don't open one/them in the backup folder. Copy to a different folder and open there. Overcautious perhaps, but mistakes happen. It's best practice to not introduce avoidable ones to the all important backup folder.


Good luck.

[kewms]

With any projects or partial projects that you find, note that you can use the File -> Import -> Scrivener Project command to tell Scrivener to extract whatever useful metadata it can from the project being imported. Results will of course depend on what you have, but this is likely to give better results than poking through the subfolders using Finder.

Katherine

[Dain]

The project was in the 2.xx format when it was last actively worked. Scrivener wants to update the format and throws several errors about sync conflicts, then this message about content that was deleted...

Screen Shot 2018-10-20 at 3.31.47 AM.png (42.98 KiB) Viewed 1368 times

[After a little digging the next day, the text of this reply was no longer relevant. I've left the image and its explanation, rather than remove the original. New details follow in the next message.]

[Dain]

With any projects or partial projects that you find, note that you can use the File -> Import -> Scrivener Project command to tell Scrivener to extract whatever useful metadata it can from the project being imported. Results will of course depend on what you have, but this is likely to give better results than poking through the subfolders using Finder.

Katherine

This fails, I am presented with instructions to open the project normally, and allow the sync conflicts to be resolved. This destroys the binder links to the various RTF files, as described in the original post.

Screen Shot 2018-10-20 at 6.41.30 PM.png (26.46 KiB) Viewed 1316 times

Now, I went digging into the 'show package contents' and it appears that my correct binder headings are still there in the binder.mob (mobile version?)

Screen Shot 2018-10-20 at 7.00.03 PM.png (65.03 KiB) Viewed 1316 times

TextWrangler opened the binder.mob file and I was able to pick out of the XML hive the scene names that are missing from the reconstructed project. It looks like everything is there. So I could, by scrubbing through the XML hive, find and recover my scene names, and the few words I feared were lost forever. They turned up in 'content.rtf' files two levels deep the mobile folder. That said I do not see a sufficiently human-readable way to trans-code this un-vanished data into something that Scrivener won't munch when it sees the old data format and updates it.

Is there a sync lock token or semaphore I could manually edit out? Are there some other things to try? Should I download an old version of Scrivener to see if it would sync the mobile changes settle any conflicts before trying to let Scrivener 3 try again?

I can see my data (and I'm working from copies) but I can't see how to get it back into play.

[Dain]

Ok, I managed to finally recover things. It wasn't really easy or clean, and I'm sure I didn't do it the way Scrivener's desinger would have. I only stopped at what worked enough to recovery the binder with all of my previously missing scenes. From there I will be able to import (as suggested in this thread) into a new project and count my blessings.

THIS REPAIR REQUIRES PROGRAMMING SOFTWARE TOOLS AND IS NOT RECOMMENDED FOR THOSE THAT DO NOT UNDERSTAND THE XML FORMAT.

Seriously, the staff here can vouch for this, you can seriously destroy your data in a hurry here if you mess with things you don't understand. My efforts are tempered by my understanding the structure of XML and programming to a degree that allows me to understand what I'm looking at. If you can't decipher 'code' then this sort of fix will be worse that doing nothing.

I opened the project.scriv package in Finder and then loaded from there both the project.scrivx and binder.mob files into my IDE (PHPStorm), then I ran the IDE's difference engine on the pair. As expected there were 20 detected differences in the files. So I started looking at those differences and judged the project container (scrivx) was correct but the mobile binder stack (binder.mob) had the correct Binder data. So I started selectively updating binder.mob with complete XML phrases (from the opening tag to the closing tag with any enclosed childern tags) from the project.scrivx file.

After making ten updates, I did not find anything else that would seem to effect function, and saved the updated binder.mob file in a new location with a different name. I then copied the project.scrivx file out of the scriv package and inserted my modified binder.mob file and renaming it to project.scrivx.

Loading the project in Scrivener3 produced a popup with a sync conflicts detection, but after that process completed I had all my data back where I was hoping to find it. I still have a number of items in the conflicts folder, but I can deal with that far easier that trying to reorder and rename my scenes by contents. Figuring out what to do in order to fix this took far longer than the few minutes I spent in PHPStorm actually fixing it.

[kewms]

What Dain said.

DO NOT try the above method unless you are FLUENT in XML.

AND have made a backup first.

There is a (slightly) less hideous method for recovering a mangled project if the "Import as Scrivener Project" approach has failed.

* Open the most recent backup you have in Scrivener. Make a backup of it to a safe location. Create a new folder in the Binder called "Recovered Versions" or something similar.

* Using Finder, locate the mangled project, right-click, and select the option to Show Package Contents. Browse to the Files/Data subfolder.

* Option-click on that folder, which will expand all the sub-folders below it. Look for files with modified dates more recent than the backup you have.

(In Scrivener 2, the project format is a little simpler. Everything will be in Files/Docs, with no subfolders.)

(If you have a lot of them, do these next steps with a few of them at a time.)

* Drag candidate files from Finder to the Recovered Versions folder in the open project.

* Inspect each one to see what it is. Assign a name in the Binder, replace the existing version, whatever seems appropriate.

* Make a backup, and look at the next file or group of files.

Eventually, you'll have incorporated all of the files from the damaged version into the backup. Make another backup and get back to work.

Katherine

[Dain]

What Dain said.

DO NOT try the above method unless you are FLUENT in XML.

AND have made a backup first.

There is a (slightly) less hideous method for recovering a mangled project if the "Import as Scrivener Project" approach has failed.

I'd have been happier going that route, I simply could not find any backups even though I know Scrivener is set to make them. Its why I felt like my only remaining option was the digital equivalent of meatball surgery.

This makes for a good example of 'when worst comes to worst' but not anything I'd recomend as a first wave of fixes.