AppleScript to Extract Bookends References into OPML for Import into Scrivener

User avatar
dglogowski
Posts: 6
Joined: Thu Feb 18, 2016 8:58 pm
Platform: Mac + iOS

Tue Aug 01, 2017 2:10 am Post

To support my research workflow, in created an AppleScript to extract selected references and associated from Bookends into an OPML file which can be imported into Scrivener's Research folder. Below is a screenshot showing the notecard structure in the binder, the top level notecard in the left editor pane, the subordinate notecards in the right editor pane, and the comments in the inspector.

Scrivener Screenshot.jpg
Scrivener Screenshot.jpg (724.81 KiB) Viewed 324 times


This script converts Bookends references and associated notes into an OPML structured file which can then be imported into Scrivener's research folder. Each reference is a top level card which contains the Title, Author, Date, Type, Publisher, Abstract, and Bookends citation key. If there are notes associated with a reference, each note creates its own subordinate note card with the Page number (if any), note header, quotes, comments, and keywoards (tags). This allows you to individually review each comment and change its status (label) within Scrivener.

The script does some error checking as follows:
    Strips images from notes
    Converts double quotes (") to single quotes (')
    Converts ampersand (&) to the word "and"

The script is written very modularly so that it can easily be adapted based on changes in OPML syntax, the need to add additional "bad characters", or changes in the Bookends reference or note delimiters or event calls. I know of one issue related to Bookends reference types and have reached out to Jon for some assistance.

Code: Select all

--Script to Export Bookends Notes to OPML file v1.0
--Written by Dave Glogowski
--29 July 2017
--
--This script converts Bookends references and associated notes into an OPML structured file which can then be imported into Scrivener's research folder.
--Each reference is a top level card which contains the Title, Author, Date, Type, Publisher, Abstract, and Bookends citation key
--If there are notes associated with a reference, each note creates its own subordinate note card with the Page number (if any), note header, quotes,
--comments, and keywoards (tags).  This allows you to individually review each comment and change its status (label) within Scrivener.
--
--The script does some error checking as follows:
--  - Strips images from notes
--  - Converts double quotes (") to single quotes (')
--  - Converts ampersand (&) to the word "and"
--
--The script is written very modularly so that it can easily be adapted based on changes in OPML syntax, the need to add additional "bad characters", or
--changes in the Bookends reference or note delimiters or event calls
--
--
--
--Variable Setup------------------------------------------------------------------------
--Set Counters
set nbr_references to 0
set nbr_notes to 0

--Set Control Variables
set remove_headers to true
set userCanceled to false

--Set Old Text Delimiters
set tid to AppleScript's text item delimiters
set note_delimiter to (ASCII character 10) & (ASCII character 10)
set comma to ","

--set Bookends Delimiters
set be_page_nbr_delimiter to "@"
set be_header_delimiter to "#"
set be_tag_delimiter to "%"
set be_quote_delimiter to ">"
set be_cite_delimiter to ";"

--set image content tags
set open_image_tag to "<iimg>"
set end_image_tag to "</iimg>"

--Set OPML Text variables
set xml_version to "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>" & return
set opml_version to "<opml version=\"1.0\">" & return
set opml_close to "</opml>" & return
set opml_header to tab & "<head>" & return
set opml_title to tab & tab & "<title>Bookends to Scrivener OPML File</title>" & return
set opml_date to tab & tab & "<dateCreated>" & (current date) & "</dateCreated>" & return
set opml_header_close to tab & "</head>" & return
set opml_body to tab & "<body>" & return
set opml_body_close to tab & "</body>" & return
set opml_outline to tab & tab & "<outline text=\""
set opml_outline_close to tab & tab & "</outline>" & return
set opml_notes to "  _note=\""

--File Name and Setup-----------------------------------------------------------------
--Ask user to remove header only Bookends notes
try
   set AlertResult to display alert "Remove Bookends Header ONLY note cards?" buttons {"No", "Yes"} default button "Yes" giving up after 5
end try

if button returned of AlertResult is "No" then set remove_headers to false


--Request file name and path from user
set target_file_name to "BE-to-OPML.opml"

try
   set data_file to choose file name with prompt "Please select destination file and folder." default name target_file_name default location (path to desktop folder)
   
on error errMsg number errNum
   if errNum = -128 then
      display dialog "User Cancelled Get File - Goodbye!"
      return false
   else
      display dialog "Try to Close Access to:" & data_file & "Error:" & errNum & return & errMsg
      close access data_file
      return false
   end if
   
end try

--Test if .opml extension was provided, if not then append to file name Actually just force .opml extension
try
   set clean_name to data_file as string
   if clean_name contains "." then
      set file_name to text 1 thru ((offset of "." in clean_name) - 1) of clean_name
      set clean_name to file_name & ".opml"
   else
      set clean_name to clean_name & ".opml"
   end if
   set data_file to clean_name
   
on error errMsg number errNum
   display dialog "Error with validaing file extension" & return & "Data Name: " & data_file & return & "Clean name: " & clean_name
   return false
end try

--Write OPML Version, Headers, and Open Body statements to File---------------
my write_to_file(xml_version, data_file, false)
my write_to_file(opml_version, data_file, true)
my write_to_file(opml_header, data_file, true)
my write_to_file(opml_title, data_file, true)
my write_to_file(opml_date, data_file, true)
my write_to_file(opml_header_close, data_file, true)
my write_to_file(opml_body, data_file, true)


--Interaction with Bookends-----------------------------------------------------------
tell application "Bookends"
   activate
   
   --get ids for selected bookends references
   set selected_ids to «event ToySRUID» "Selection"
   
   --test to make sure user selected Bookends references
   if selected_ids is "" then
      display dialog "No Boookends References were selected." & return & return & "Please select 1 or more references and restrart"
      return false
   end if
   
   set selected_ids to words of selected_ids
   
   --get the cite_keys from Bookends
   set the clipboard to ""
   tell application "System Events" to keystroke "y" using command down
   delay 0.05
   set be_cite_key to the clipboard
   
   --strip citation key brackets
   set cite_length to length of be_cite_key
   set cite_list to text 2 thru (cite_length - 1) of be_cite_key
   set AppleScript's text item delimiters to be_cite_delimiter
   set cite_keys to text items of cite_list
   set AppleScript's text item delimiters to tid
   
   --Process Handiling For Each Reference------------------------------------------
   --get citation, quotes, comments, and tags for of the selected references
   repeat with i from 1 to length of selected_ids
      
      --set variables and counters
      set ref_id to item i of selected_ids
      set ref_nbr to ref_id
      set nbr_references to nbr_references + 1
      
      --For each reference build the top level OPML outline
      --with the Author, Date, Bookends Reference Number, Title, Abstract, etc
      
      --Author and Date Processing
      --extract author and dates from citation key
      set cite_key_item to text item i of cite_keys
      set AppleScript's text item delimiters to comma
      set cite_key_components to text items of cite_key_item
      set AppleScript's text item delimiters to tid
      set key_component_count to count of items in cite_key_components
      
      set no_date to false
      set ref_date to «event ToySRFLD» ref_id given string:"thedate"
      if ref_date is "" then set no_date to true
      
      if key_component_count = 3 then
         set ref_author to first item of cite_key_components
         set ref_date to second item of cite_key_components
      end if
      
      if key_component_count = 2 then
         if no_date then
            set ref_author to first item of cite_key_components
            set ref_date to " Undated"
         else
            set ref_author to "No Author(s)"
            set ref_date to first item of cite_key_components
         end if
      end if
      
      if key_component_count = 1 then
         set ref_author to "No Author(s)"
         set ref_date to " Undated"
      end if
      
      --strip leading blank space from cite key, author and date      
      if first character of ref_author is space then set ref_author to characters 2 thru -1 of ref_author
      if first character of ref_date is space then set ref_date to characters 2 thru -1 of ref_date
      if first character of cite_key_item is space then set cite_key_item to characters 2 thru -1 of cite_key_item
      
      set ref_author to my replace_bad_characters(ref_author)
      
      --Index Card Processing (Title, Journal, etc)
      
      --Reference Title
      set ref_title to «event ToySRFLD» ref_id given string:"title"
      if ref_title is "" then
         set ref_title to "No Title"
      else
         set ref_title to my replace_bad_characters(ref_title)
      end if
      
      --Reference Type (Journal, Book, etc)
      --set ref_type to "undefined type"
      
      set ref_type to «event ToySGUID» ref_id given string:"RIS"
      if ref_type is "" then
         set ref_type to "undefined type"
      else
         set ref_type to second word of ref_type as text
         set ref_type to my replace_bad_characters(ref_type)
      end if
      
      --Reference Publisher
      set ref_publisher to «event ToySRFLD» ref_id given string:"publisher"
      if ref_publisher is "" then
         set ref_publisher to "undefined publisher"
      else
         set ref_publisher to my replace_bad_characters(ref_publisher)
      end if
      
      --Reference Abstract
      set ref_abstract to «event ToySRFLD» ref_id given string:"abstract"
      if ref_abstract is "" then
         set ref_abstract to "Abstract not provided"
      else
         set ref_abstract to my replace_bad_characters(ref_abstract)
      end if
      
      --Form the Reference Card OPML Outline Statement
      set ref_text to opml_outline & ref_author & ", " & ref_date & " (ID:" & ref_nbr & ")\"" & ¬
         opml_notes & ref_title & return & ¬
         ref_author & return & ¬
         ref_date & "   " & ref_type & " - " & ref_publisher & return & ¬
         "------" & return & ¬
         "Abstract: " & return & ref_abstract & "\">" & return as text
      
      my write_to_file(ref_text, data_file, true)
      
      --Process Handiling For Each Note within Each Reference------------      
      --get notes from this reference
      set ref_notes to «event ToySRFLD» ref_id given string:"notes"
      
      --extract each note and create separate note card (subordinate outline)      
      set AppleScript's text item delimiters to note_delimiter
      set ref_notes to text items of ref_notes
      set AppleScript's text item delimiters to tid
      
      repeat with p from 1 to length of ref_notes
         --reset variables
         set header_only to false
         set keywords to false
         set quotes to false
         set ref_note_header to " "
         set ref_page_nbr to "##"
         set ref_note_title to " "
         set ref_note_text to " "
         set ref_tags to " "
         set ref_quote to " "
         set ref_note_quote to " "
         set ref_note_comment to ""
         
         -- ref_note_item is an individual note within the note stream
         set ref_note_item to item p of ref_notes
         
         --parse note_item for Bookends header and page number
         if ref_note_item is not "" then
            
            set ref_note_list to paragraphs of ref_note_item
            
            --test and process headers   
            if first character in ref_note_item is be_header_delimiter then
               --header
               set ref_note_header to first paragraph in ref_note_item
               --determine if header only and set note contents to rest of notes
               if (count of ref_note_list) > 1 then
                  set ref_note_text to text (second paragraph of ref_note_item) thru -1 of ref_note_item
               else
                  set ref_note_text to "Header Only"
                  if remove_headers is true then set header_only to true
               end if
               --set header, but first test if header also includes page number
               if second character in ref_note_item is be_page_nbr_delimiter then
                  --with page number
                  set ref_page_nbr to "@" & first word of ref_note_item
                  set ref_note_header to text (second word of ref_note_header) thru -1 of ref_note_header
               else
                  --without page number
                  set ref_note_header to text (first word of ref_note_header) thru -1 of ref_note_header
               end if
            else
               --no header, set title to untitled note and set contents to all of note
               set ref_note_header to "Untitled Note"
               set ref_note_text to ref_note_item
            end if
            
            --test for page numbers      
            if first character in ref_note_item is be_page_nbr_delimiter then
               set ref_page_nbr to "@" & first word of ref_note_item
               set ref_note_text to text (second word of ref_note_item) thru -1 of ref_note_item
            end if
            
            --form the note card title
            set ref_note_title to ref_page_nbr & " - " & ref_note_header
            
            --test note title for well formed contents (bad opml characters)
            set ref_note_title to my replace_bad_characters(ref_note_title)
            
            --Process Handling For Each Line (Paragraph) within Each Note
            --extract each line (paragraph) of the note      
            set AppleScript's text item delimiters to (ASCII character 10)
            set ref_note_text to text items of ref_note_text
            set AppleScript's text item delimiters to tid
            
            repeat with n from 1 to length of ref_note_text
               
               --get and process each paragraph (segement) of the note
               set ref_note_body to item n of ref_note_text
               
               --test notes for well formed contents (ie. no images)
               if (open_image_tag is in ref_note_body) then
                  set image_start to (offset of open_image_tag in ref_note_body)
                  set image_end to (offset of end_image_tag in ref_note_body) + (length of end_image_tag) - 1
                  set first_half to text 1 thru image_start of ref_note_body
                  set last_half to text from image_end to -1 of ref_note_body
                  set ref_note_body to first_half & " -- Graphics Image Removed --" & last_half
               end if
               
               --test notes for well formed contents (bad opml characters)
               set ref_note_body to my replace_bad_characters(ref_note_body)
               
               if ref_note_body is not "" then
                  
                  --clear temp vars
                  set temp_tag to ""
                  set temp_quote to ""
                  set no_comment_flag to false
                  
                  --test for tags
                  if first character in ref_note_body is be_tag_delimiter then
                     set temp_tags to ref_note_body
                     set AppleScript's text item delimiters to be_tag_delimiter
                     set temp_tags to text items of temp_tags
                     set AppleScript's text item delimiters to space
                     set temp_tags to temp_tags as text
                     set AppleScript's text item delimiters to tid
                     set ref_tags to ref_tags & temp_tags
                     set no_comment_flag to true
                     set keywords to true
                  end if
                  
                  
                  --test for bookends quotes            
                  if first character in ref_note_body is be_quote_delimiter then
                     set temp_quote to ref_note_body
                     set temp_quote to text (second character of temp_quote) thru -1 of temp_quote
                     set ref_note_quote to ref_quote & temp_quote
                     set no_comment_flag to true
                     set quotes to true
                  end if
                  
                  --form note comments
                  if no_comment_flag is false then ¬
                     set ref_note_comment to ref_note_comment & ref_note_body & return
                  
               end if
               
            end repeat
            
            --form the note card OPML statements
            set ref_key to "Keywords: " & ref_tags & return & return
            set ref_quote to "Quote(s): " & return & ref_note_quote & return & return
            set ref_comment to "Comments: " & return & ref_note_comment & return
            if ref_page_nbr is "##" then
               set ref_cite_key to "Citation Key: {" & cite_key_item & "}" & return
            else
               set ref_cite_key to "Citation Key: {" & cite_key_item & ref_page_nbr & "}" & return
            end if
            
            set note_card to "" as text
            if keywords is true then set note_card to note_card & ref_key
            if quotes is true then set note_card to note_card & ref_quote
            set note_card to note_card & ref_comment & "----------" & return & ref_cite_key
            
            set ref_note_contents to tab & opml_outline & ref_note_title & "\"" & opml_notes & note_card & "\"/>" & return as text
            
            --write contents of note
            if header_only is false then
               my write_to_file(ref_note_contents, data_file, true)
               set nbr_notes to nbr_notes + 1
            end if
         end if
      end repeat
      my write_to_file(opml_outline_close, data_file, true)
   end repeat
end tell

my write_to_file(opml_body_close, data_file, true)
my write_to_file(opml_close, data_file, true)

display notification "Complete" & return & return & "Exported " & nbr_references & " References and " & nbr_notes & " Notes"
return true

--end of script **************************************************

--subroutine area ************************************************
--write_to_this_file subroutine
on write_to_file(this_data, target_file, append_data)
   try
      set target_file to target_file as string
      set open_target_file to open for access target_file with write permission
      if append_data is false then set eof of the open_target_file to 0
      write this_data as «class utf8» to open_target_file starting at eof
      close access open_target_file
      return true
      
   on error errMsg number errNum
      if errNum = -49 then
         close access target_file
         set open_target_file to open for access target_file with write permission
         if append_data is false then set eof of the open_target_file to 0
         write this_data as «class utf8» to open_target_file starting at eof
         close access open_target_file
         return true
      else
         display dialog "Write Subroutine Error:" & return & "Target_File: " & target_file & return & "Open_Target_File: " & open_target_file & return & "Error: " & errNum & " - " & errMsg
         close access open_target_file
         return false
      end if
   end try
end write_to_file


--test for well formed contents (bad opml characters) subroutine
on replace_bad_characters(input_string)
   try
      --set arrays for error checking
      set bad_opml_char_array to {"\"", "&"}
      set good_opml_char_array to {"'", "and"}
      set input_string to input_string as string
      set output_string to ""
      set good_char to ""
      
      repeat with x from 1 to count of characters in input_string
         set good_char to character x of input_string
         repeat with y from 1 to length of bad_opml_char_array
            if good_char is equal to item y of bad_opml_char_array then set good_char to item y of good_opml_char_array
         end repeat
         set output_string to output_string & good_char
      end repeat
      return output_string
      
   on error errMsg number errNum
      display dialog "Error replacing bad OPML characters.  Error Number: " & errNum & " - " & errMsg
      return false
   end try
end replace_bad_characters
Cheers,
Dave

User avatar
nontroppo
Posts: 752
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Tue Aug 01, 2017 5:52 am Post

First off, many thanks for sharing what could be a great tool. Sadly this isn't working for me, as it depends on a user using Bookends default temporary citation format (I use another format).

In addition, you use System Events to inject a keyboard command CMD+Y to "copy" the temporary citations, but if Scrivener is linked it will also paste into Scrivener. This will delete any selected text, so a user has to be aware of this before running the script...

:idea: There is a solution to both of these issues! Why don't you use «event ToySGUID» (p.23 of the BE manual) to get a formatted citation which you can reliably parse. You do not need to use CMD+Y and depend on a clipboard temporary citation in a specific format. You pass the unique ID (you already extract using «event ToySRUID») and a format (you can choose anything you'd find easy to parse), and it will return UTF8 text (or RTF but that would be incompatible with OPML I think). This would make the script more robust.

User avatar
dglogowski
Posts: 6
Joined: Thu Feb 18, 2016 8:58 pm
Platform: Mac + iOS

Thu Aug 03, 2017 2:52 pm Post

Thanks for the comments and feedback. I've updated the script to remove the keyboard command (CMD-Y) which caused the unintended consequence of overwriting your Scrivener text. The revised script is provided at the bottom of this reply.

I use the default temporary citation format as its the most common. The rationale for including the temporary citation is this allows me to work off my iPad in Scrivener and insert the citation keys by copying them from the notecard into the text. As for the your use of another temporary citation format, I contacted Jon, the developer of Bookends, to address this issue. Basically the format of the temporary citation does not matter to Bookends as long as it can resolve it a specific reference in Bookends. Since I provide the Bookends Unique ID, the rest is merely for our readability. Additionally, as Jon notes in his reply (quoted below), you can always do a Proofreading Scan (see Bookends User Guide v12.7, pg 225-226) to convert the temporary citation to your preferred temporary citation prior to doing a full Scan.

There are three options to create the temporary citation. They all work, it's a matter of personal preference:

Author, Date, Unique ID
By Content (Bookends creates a mixture of fields that unambiguously identifies a reference)
By Format (whatever the user designs)

If they don't like Author, Date, Unique ID (which is generally preferred) they can do a Proofreading scan at any time to convert them to the preference they have chosen.

Is this for creating the temp citation of Author, Date, Unique ID also? If so, the contents of the date field are what you want, not the year. When Bookends scans it matches the contents of the temp citation with the reference's contents. An exception so this rule if that if you specify the Unique ID the rest of the citation is simple for the user's benefit -- it's ignored since the Unique ID contains all the information Bookends needs to make an unambiguous match.


I hope this helps. Again, I greatly appreciate the feedback to make this a useful tool.

Code: Select all

--Script to Export Bookends Notes to OPML file v1.0
--Written by Dave Glogowski
--29 July 2017
--
--This script converts Bookends references and associated notes into an OPML structured file which can then be imported into Scrivener's research folder.
--Each reference is a top level card which contains the Title, Author, Date, Type, Publisher, Abstract, and Bookends citation key
--If there are notes associated with a reference, each note creates its own subordinate note card with the Page number (if any), note header, quotes,
--comments, and keywoards (tags).  This allows you to individually review each comment and change its status (label) within Scrivener.
--
--The script does some error checking as follows:
--  - Strips images from notes
--  - Converts double quotes (") to single quotes (')
--  - Converts ampersand (&) to the word "and"
--
--The script is written very modularly so that it can easily be adapted based on changes in OPML syntax, the need to add additional "bad characters", or
--changes in the Bookends reference or note delimiters or event calls
--
--
--
--Variable Setup------------------------------------------------------------------------
--Set Counters
set nbr_references to 0
set nbr_notes to 0

--Set Control Variables
set remove_headers to true
set userCanceled to false

--Set Old Text Delimiters
set tid to AppleScript's text item delimiters
set note_delimiter to (ASCII character 10) & (ASCII character 10)
set comma to ","
set date_separators to {"/", " ", ".", "-"}
set digits to {"0", "1", "2", "3", "4", "5", "6", "7", "8", "9"}
set era_first_digit to {"1", "2"}


--set Bookends Delimiters
set be_page_nbr_delimiter to "@"
set be_header_delimiter to "#"
set be_tag_delimiter to "%"
set be_quote_delimiter to ">"
set be_cite_delimiter to ";"

--set image content tags
set open_image_tag to "<iimg>"
set end_image_tag to "</iimg>"

--Set OPML Text variables
set xml_version to "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>" & return
set opml_version to "<opml version=\"1.0\">" & return
set opml_close to "</opml>" & return
set opml_header to tab & "<head>" & return
set opml_title to tab & tab & "<title>Bookends to Scrivener OPML File</title>" & return
set opml_date to tab & tab & "<dateCreated>" & (current date) & "</dateCreated>" & return
set opml_header_close to tab & "</head>" & return
set opml_body to tab & "<body>" & return
set opml_body_close to tab & "</body>" & return
set opml_outline to tab & tab & "<outline text=\""
set opml_outline_close to tab & tab & "</outline>" & return
set opml_notes to "  _note=\""

--File Name and Setup-----------------------------------------------------------------
--Ask user to remove header only Bookends notes
try
   set AlertResult to display alert "Remove Bookends Header ONLY note cards?" buttons {"No", "Yes"} default button "Yes" giving up after 5
end try

if button returned of AlertResult is "No" then set remove_headers to false


--Request file name and path from user
set target_file_name to "BE-to-OPML.opml"

try
   set data_file to choose file name with prompt "Please select destination file and folder." default name target_file_name default location (path to desktop folder)
   
on error errMsg number errNum
   if errNum = -128 then
      display dialog "User Cancelled Get File - Goodbye!"
      return false
   else
      display dialog "Try to Close Access to:" & data_file & "Error:" & errNum & return & errMsg
      close access data_file
      return false
   end if
   
end try

--Test if .opml extension was provided, if not then append to file name Actually just force .opml extension
try
   set clean_name to data_file as string
   if clean_name contains "." then
      set file_name to text 1 thru ((offset of "." in clean_name) - 1) of clean_name
      set clean_name to file_name & ".opml"
   else
      set clean_name to clean_name & ".opml"
   end if
   set data_file to clean_name
   
on error errMsg number errNum
   display dialog "Error with validaing file extension" & return & "Data Name: " & data_file & return & "Clean name: " & clean_name
   return false
end try

--Write OPML Version, Headers, and Open Body statements to File---------------
my write_to_file(xml_version, data_file, false)
my write_to_file(opml_version, data_file, true)
my write_to_file(opml_header, data_file, true)
my write_to_file(opml_title, data_file, true)
my write_to_file(opml_date, data_file, true)
my write_to_file(opml_header_close, data_file, true)
my write_to_file(opml_body, data_file, true)


--Interaction with Bookends-----------------------------------------------------------
tell application "Bookends"
   activate
   
   --get ids for selected bookends references
   set selected_ids to «event ToySRUID» "Selection"
   
   --test to make sure user selected Bookends references
   if selected_ids is "" then
      display dialog "No Boookends References were selected." & return & return & "Please select 1 or more references and restrart"
      return false
   end if
   
   set selected_ids to words of selected_ids
   
   --Process Handiling For Each Reference------------------------------------------
   --get citation, quotes, comments, and tags for of the selected references
   repeat with i from 1 to length of selected_ids
      
      --set variables and counters
      set ref_id to item i of selected_ids
      set ref_nbr to ref_id
      set nbr_references to nbr_references + 1
      
      --FOR EACH REFERENCE BUILD THE TOP LEVEL OPML OUTLINE
      
      --Reference Author or Editor      
      set ref_author to «event ToySRFLD» ref_id given string:"authors"
      if ref_author is "" then set ref_author to «event ToySRFLD» ref_id given string:"editors"
      
      if ref_author is "" then
         set ref_title to "No Authors or Editors"
      else
         set AppleScript's text item delimiters to linefeed
         set author_list to every text item of ref_author
         set author_list_count to length of author_list
         set AppleScript's text item delimiters to comma
         set first_author to (first item of author_list as string)
         set remaining_authors to every text item of first_author
         if (last character of first_author) is comma then set first_author to first_author's text 1 thru -2
         if author_list_count = 1 then
            set ref_author to (first item of remaining_authors as string)
         else if author_list_count = 2 then
            set second_author to (second item of author_list as string)
            set final_authors to every text item of second_author
            set ref_author to (first item of remaining_authors as string) & " and " & (first item of final_authors as string)
         else if author_list_count > 2 then
            set ref_author to (first item of remaining_authors as string) & " et al."
         end if
         set ref_author to my replace_bad_characters(ref_author)
      end if
      set AppleScript's text item delimiters to tid
      
      
      --Reference Year      
      set ref_date to «event ToySRFLD» ref_id given string:"thedate"
      if ref_date is "" then
         set ref_date to "Undated"
      else
         set ref_date to my replace_bad_characters(ref_date)
         set got_year to false
         repeat with d in date_separators
            set AppleScript's text item delimiters to d
            set date_list to every text item of ref_date
            set AppleScript's text item delimiters to tid
            repeat with k in date_list
               if length of k is 4 then
                  if first character of k is in era_first_digit then
                     set ref_year to k
                     set got_year to true
                  end if
               end if
               if got_year then exit repeat
            end repeat
            if got_year then exit repeat
         end repeat
         if got_year then set ref_date to ref_year
      end if
      
      
      --Reference Title
      set ref_title to «event ToySRFLD» ref_id given string:"title"
      if ref_title is "" then
         set ref_title to "No Title"
      else
         set ref_title to my replace_bad_characters(ref_title)
      end if
      
      
      --Reference Type (Journal, Book, etc)
      set ref_type to «event ToySGUID» ref_id given string:"RIS"
      if ref_type is "" then
         set ref_type to "undefined type"
      else
         set ref_type to second word of ref_type as text
         set ref_type to my replace_bad_characters(ref_type)
      end if
      
      
      --Reference Publisher
      set ref_publisher to «event ToySRFLD» ref_id given string:"publisher"
      if ref_publisher is "" then
         set ref_publisher to "undefined publisher"
      else
         set ref_publisher to my replace_bad_characters(ref_publisher)
      end if
      
      
      --Reference Abstract
      set ref_abstract to «event ToySRFLD» ref_id given string:"abstract"
      if ref_abstract is "" then
         set ref_abstract to "Abstract not provided"
      else
         set ref_abstract to my replace_bad_characters(ref_abstract)
      end if
      
      
      --Build Cite Key
      set cite_key to ref_author & ", " & ref_date & ", #" & ref_nbr
      
      
      --Form the Reference Card OPML Outline Statement
      set ref_text to opml_outline & ref_author & ", " & ref_date & " (ID:" & ref_nbr & ")\"" & ¬
         opml_notes & ref_title & return & ¬
         ref_author & return & ¬
         ref_date & "   " & ref_type & " - " & ref_publisher & return & ¬
         "------" & return & ¬
         "Abstract: " & return & ref_abstract & "\">" & return as text
      
      my write_to_file(ref_text, data_file, true)
      
      --Process Handiling For Each Note within Each Reference------------      
      --get notes from this reference
      set ref_notes to «event ToySRFLD» ref_id given string:"notes"
      
      --extract each note and create separate note card (subordinate outline)      
      set AppleScript's text item delimiters to note_delimiter
      set ref_notes to text items of ref_notes
      set AppleScript's text item delimiters to tid
      
      repeat with p from 1 to length of ref_notes
         --reset variables
         set header_only to false
         set keywords to false
         set quotes to false
         set ref_note_header to " "
         set ref_page_nbr to "##"
         set ref_note_title to " "
         set ref_note_text to " "
         set ref_tags to " "
         set ref_quote to " "
         set ref_note_quote to " "
         set ref_note_comment to ""
         
         -- ref_note_item is an individual note within the note stream
         set ref_note_item to item p of ref_notes
         
         --parse note_item for Bookends header and page number
         if ref_note_item is not "" then
            
            set ref_note_list to paragraphs of ref_note_item
            
            --test and process headers   
            if first character in ref_note_item is be_header_delimiter then
               --header
               set ref_note_header to first paragraph in ref_note_item
               --determine if header only and set note contents to rest of notes
               if (count of ref_note_list) > 1 then
                  set ref_note_text to text (second paragraph of ref_note_item) thru -1 of ref_note_item
               else
                  set ref_note_text to "Header Only"
                  if remove_headers is true then set header_only to true
               end if
               --set header, but first test if header also includes page number
               if second character in ref_note_item is be_page_nbr_delimiter then
                  --with page number
                  set ref_page_nbr to "@" & first word of ref_note_item
                  set ref_note_header to text (second word of ref_note_header) thru -1 of ref_note_header
               else
                  --without page number
                  set ref_note_header to text (first word of ref_note_header) thru -1 of ref_note_header
               end if
            else
               --no header, set title to untitled note and set contents to all of note
               set ref_note_header to "Untitled Note"
               set ref_note_text to ref_note_item
            end if
            
            --test for page numbers      
            if first character in ref_note_item is be_page_nbr_delimiter then
               set ref_page_nbr to "@" & first word of ref_note_item
               set ref_note_text to text (second word of ref_note_item) thru -1 of ref_note_item
            end if
            
            --form the note card title
            set ref_note_title to ref_page_nbr & " - " & ref_note_header
            
            --test note title for well formed contents (bad opml characters)
            set ref_note_title to my replace_bad_characters(ref_note_title)
            
            --Process Handling For Each Line (Paragraph) within Each Note
            --extract each line (paragraph) of the note      
            set AppleScript's text item delimiters to (ASCII character 10)
            set ref_note_text to text items of ref_note_text
            set AppleScript's text item delimiters to tid
            
            repeat with n from 1 to length of ref_note_text
               
               --get and process each paragraph (segement) of the note
               set ref_note_body to item n of ref_note_text
               
               --test notes for well formed contents (ie. no images)
               if (open_image_tag is in ref_note_body) then
                  set image_start to (offset of open_image_tag in ref_note_body)
                  set image_end to (offset of end_image_tag in ref_note_body) + (length of end_image_tag) - 1
                  set first_half to text 1 thru image_start of ref_note_body
                  set last_half to text from image_end to -1 of ref_note_body
                  set ref_note_body to first_half & " -- Graphics Image Removed --" & last_half
               end if
               
               --test notes for well formed contents (bad opml characters)
               set ref_note_body to my replace_bad_characters(ref_note_body)
               
               if ref_note_body is not "" then
                  
                  --clear temp vars
                  set temp_tag to ""
                  set temp_quote to ""
                  set no_comment_flag to false
                  
                  --test for tags
                  if first character in ref_note_body is be_tag_delimiter then
                     set temp_tags to ref_note_body
                     set AppleScript's text item delimiters to be_tag_delimiter
                     set temp_tags to text items of temp_tags
                     set AppleScript's text item delimiters to space
                     set temp_tags to temp_tags as text
                     set AppleScript's text item delimiters to tid
                     set ref_tags to ref_tags & temp_tags
                     set no_comment_flag to true
                     set keywords to true
                  end if
                  
                  
                  --test for bookends quotes            
                  if first character in ref_note_body is be_quote_delimiter then
                     set temp_quote to ref_note_body
                     set temp_quote to text (second character of temp_quote) thru -1 of temp_quote
                     set ref_note_quote to ref_quote & temp_quote
                     set no_comment_flag to true
                     set quotes to true
                  end if
                  
                  --form note comments
                  if no_comment_flag is false then ¬
                     set ref_note_comment to ref_note_comment & ref_note_body & return
                  
               end if
               
            end repeat
            
            --form the note card OPML statements
            set ref_key to "Keywords: " & ref_tags & return & return
            set ref_quote to "Quote(s): " & return & ref_note_quote & return & return
            set ref_comment to "Comments: " & return & ref_note_comment & return
            if ref_page_nbr is "##" then
               set ref_cite_key to "Citation Key: {" & cite_key & "}" & return
            else
               set ref_cite_key to "Citation Key: {" & cite_key & ref_page_nbr & "}" & return
            end if
            
            set note_card to "" as text
            if keywords is true then set note_card to note_card & ref_key
            if quotes is true then set note_card to note_card & ref_quote
            set note_card to note_card & ref_comment & "----------" & return & ref_cite_key
            
            set ref_note_contents to tab & opml_outline & ref_note_title & "\"" & opml_notes & note_card & "\"/>" & return as text
            
            --write contents of note
            if header_only is false then
               my write_to_file(ref_note_contents, data_file, true)
               set nbr_notes to nbr_notes + 1
            end if
         end if
      end repeat
      my write_to_file(opml_outline_close, data_file, true)
   end repeat
end tell

my write_to_file(opml_body_close, data_file, true)
my write_to_file(opml_close, data_file, true)

display notification "Complete" & return & return & "Exported " & nbr_references & " References and " & nbr_notes & " Notes"
return true

--end of script **************************************************

--subroutine area ************************************************
--write_to_this_file subroutine
on write_to_file(this_data, target_file, append_data)
   try
      set target_file to target_file as string
      set open_target_file to open for access target_file with write permission
      if append_data is false then set eof of the open_target_file to 0
      write this_data as «class utf8» to open_target_file starting at eof
      close access open_target_file
      return true
      
   on error errMsg number errNum
      if errNum = -49 then
         close access target_file
         set open_target_file to open for access target_file with write permission
         if append_data is false then set eof of the open_target_file to 0
         write this_data as «class utf8» to open_target_file starting at eof
         close access open_target_file
         return true
      else
         display dialog "Write Subroutine Error:" & return & "Target_File: " & target_file & return & "Open_Target_File: " & open_target_file & return & "Error: " & errNum & " - " & errMsg
         close access open_target_file
         return false
      end if
   end try
end write_to_file


--test for well formed contents (bad opml characters) subroutine
on replace_bad_characters(input_string)
   try
      --set arrays for error checking
      set bad_opml_char_array to {"\"", "&"}
      set good_opml_char_array to {"'", "and"}
      set input_string to input_string as string
      set output_string to ""
      set good_char to ""
      
      repeat with x from 1 to count of characters in input_string
         set good_char to character x of input_string
         repeat with y from 1 to length of bad_opml_char_array
            if good_char is equal to item y of bad_opml_char_array then set good_char to item y of good_opml_char_array
         end repeat
         set output_string to output_string & good_char
      end repeat
      return output_string
      
   on error errMsg number errNum
      display dialog "Error replacing bad OPML characters.  Error Number: " & errNum & " - " & errMsg
      return false
   end try
end replace_bad_characters
Cheers,
Dave

User avatar
nontroppo
Posts: 752
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Fri Aug 04, 2017 2:18 am Post

Dave, this new version is now working for me, thank you!!! I agree it makes sense to use the default temp citation.

However, I do not see any temp citation keys in the OPML output. The reason is that you only copy the citation key into the Notes subitem, but what if references do not have notes?

I was also thinking of a way to include links back to Bookends. Bookends allow a URI like bookends://sonnysoftware.com/28593 — but I think we cannot mark up a link in the OPML XML and so we would need Scrivener to do something "special" with the OPML import. A Scrivener wish list item for me would be to allow links in the OPML import:

Code: Select all

<outline text="Doe et al., 2017 (ID:28593)"  link="bookends://sonnysoftware.com/28593"_note="..."/></outline>


Scrivener could then make this a resolvable RTF URI, click the link in Scrivener and go straight back to Bookends ref directly! Note I made some scripts which allows you to select a uniqueID or temp citation in Scrivener (or any other software) and open it in bookends as a workaround: https://www.sonnysoftware.com/phpBB3/vi ... f=6&t=3995

User avatar
dglogowski
Posts: 6
Joined: Thu Feb 18, 2016 8:58 pm
Platform: Mac + iOS

Fri Aug 04, 2017 2:11 pm Post

I did not include the temporary citation in the top-level note card mainly to reduce space. Also it did not necessarily fit within my note taking construct. Basically, if a reference does not have any note cards, then it has not been read yet or was not significant. In either case, I would not be discussing it in my writings and hence no need to cite it.

Now having said that, this script can easily be tailored to include the temporary citation on the top-level card.

As for the links, check over on the Bookends forum. A user there as already added the DOI and URL to the script. They did not post the code, but I have asked for it since I am intrigued as to how he or she got the data item.
Cheers,
Dave

User avatar
nontroppo
Posts: 752
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Fri Aug 04, 2017 4:54 pm Post

That user is me! I've added PMID, DOI (very important for sciences) and user1/BibTeX key. I've posted my modifications to a gist:

See the Code
-
Direct download

I've tried several methods to try to convince Scrivener to convert the URLs into live hyperlinks, but to no avail. You can manually do it (press space after URL in Scrivener editor, autocomplete does the rest). We'll need to wishlist Keith to get some custom parsing when Scrivener imports...

User avatar
nontroppo
Posts: 752
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Sat Aug 05, 2017 3:26 am Post

I made a formal wish list request to try improve the link support for the OPML import from Bookends:

viewtopic.php?f=4&t=38888&p=236167#p236167

I also suggested a keywords attribute — it would be awesome if we could get the keywords linked to a reference from Bookends converted to Scrivener keywords!!!

User avatar
dglogowski
Posts: 6
Joined: Thu Feb 18, 2016 8:58 pm
Platform: Mac + iOS

Sat Aug 05, 2017 1:20 pm Post

Sorry the delay, but that work thing kept rearing its ugly head yesterday.

And here I though I was help to connect two discrete users to solve a problem, when in fact they were in perfect alignment -- you.

As for the links, completely agree with the ability to click on a Bookends link within Scrivener in order to get back to the full reference and source material; especially if it can still work within the iOS version.

I will look at the updated code later this evening. I will reply over on the Bookends forum since we have other interested parties looking at the code.
Cheers,
Dave

User avatar
nontroppo
Posts: 752
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Sun Sep 10, 2017 9:53 am Post

Hi Dave, I've integrated your script into an Alfred workflow of 8 tools for Bookends, with several of these tools make interfacing with Scrivener simpler:

https://github.com/iandol/bookends-tools

The source of the latest version is available at https://github.com/iandol/bookends-tool ... pplescript — to fit better with Alfred or other tools, I've converted it into a command-line script where you pass the path to output to (default is user's Desktop)