CSV Import

Ch
ChiaLynn
Posts: 38
Joined: Wed Nov 16, 2011 2:11 am
Platform: Mac
Contact:

Wed Mar 28, 2012 11:49 pm Post

Hi, Matt (and friends!).

I'm hoping you can give me an example of what the comma-separated values should look like for a successful import. I'm getting a pop-up box that says "Imported O of (x) rows," where (x) is the number of rows in the CSV.

That's the short version. The much longer version follows, but feel free to skip it if you don't need that much detail.

After spending the afternoon manually copying the events I had in the Timeline Project's Timeline (http://thetimelineproj.sourceforge.net/) into Aeon Timeline, I discovered that Timeline's file format (.timeline) is actually XML. "That would have been handy to know," I thought. "I should figure out how to import that into Aeon Timeline so I can write a quick tutorial, just in case anybody else tries to do the same thing."

I've managed to run the XML through Excel (Numbers should also work, though I have to convince OS X that my OmniOutliner trial doesn't have a monopoly on XMLs before I can try it) and export it, first with tab-separated values, then with comma-separated values. When I try to import it, though, in either format, Aeon tells me it "Imported 0 of 0 rows."

Adjusting the date format hasn't helped. I strongly suspect that I don't have the correct values in the start and end date fields, but I'm not sure what they should look like.

This is what one event looked like when it came out of Timeline:

<events>
<event>
<start>1940-5-14 12:59:0</start>
<end>1940-5-14 12:59:0</end>
<text>Fall of Holland</text>
<fuzzy>False</fuzzy>
<locked>False</locked>
<ends_today>False</ends_today>
<category>The War on the Continent</category>
</event>

After I converted it to .csv, I got rid of the <events> tag there at the top, as well as a lot of cruft specific to the way Timeline tags and codes files, and changed the start and end fields:

<start date>1940-5-14 12:59:0</start date>
<end date>1940-5-14 12:59:0</end date>
<event title>Fall of Holland</text>
<fuzzy>False</fuzzy>
<locked>False</locked>
<ends_today>False</ends_today>
<category>The War on the Continent</category>

Same result.

I then tried stripping out everything except <start date>, <end date>, and <event title>. This time, I got "Imported 0 of 3 rows."

I've tried reformatting the event date and time, based on an .aeonxml I opened in Excel, but again with the same result. I also tried converting an .aeonxml into a .csv and importing that, but when I did, I didn't get any kind of message at all - the file dialogue closes, and there's the same blank timeline I started with.

As I said, I'm quite sure it's something I'm doing wrong, but if someone could help me figure out what, I'd appreciate it.

Thanks!
Chia

ma
matt
Posts: 1180
Joined: Mon Jul 30, 2007 9:35 am

Thu Mar 29, 2012 2:18 am Post

Hi,
If you email me a copy of the file you are trying to import to support (AT) scribblecode (DOT) com, I will take a look at it and work out an efficient way to convert to the correct import format -- and test if there is anything I can do to make the csv import easier at the same time.

Matt

Ch
ChiaLynn
Posts: 38
Joined: Wed Nov 16, 2011 2:11 am
Platform: Mac
Contact:

Thu Mar 29, 2012 2:32 am Post

Wonderful, thank you! I'll send it to you tomorrow.

ma
matt
Posts: 1180
Joined: Mon Jul 30, 2007 9:35 am

Sat Mar 31, 2012 7:26 am Post

The file format you eventually need to get the data into is as follows:

Start Date, End Date, Event Title, Arc, Label (the last 2 fields are optional)

The fields can be separated by commas or tabs.

So what you are trying to create is something like this:

02/03/2002 04:04:15,02/03/2002 04:04:15,First Event,Global,Midnight
08/05/2002 06:04:15,09/05/2002 06:04:15,Second Event,Global,Purple
13/09/2002 07:04:15,13/09/2002 07:04:15,Third Event,Arc 2,Midnight

The steps from the XML format to the CSV format are not especially straightforward though, unfortunately. At some point, you need to go from having the data on separate lines for each piece of data to having all data for each event on one line.

Matt

mw
mwra
Posts: 21
Joined: Thu Jun 24, 2010 1:11 pm
Platform: Mac
Location: Portsmouth, UK

Fri Jun 01, 2012 6:12 pm Post

CSV fails (silently) if - as is normal with CSV file - you include the target field names as line #1. I'd suggest the import filter ignore line #1 if it doesn't stat with a date. It isn't necessary to include the time element of the data. CSV import fails if any all values are quote-enclosed; this is an error - if nothing else if is the way a CSV field is used to delimit a value contain a comment. Currently you can't import a title like 'Hello, world' as it contains a comma.

Tab-delim import is ... broken. As with CSV it fails completely if the filed names are included in line #1. That said, without a standard line #1 all I ever get for the event is Start Date and nothing else if End Date isn't supplied. If it is supplied it is imported as the event title! I couldn't test the other fields.

My polite suggestion would be to disable the import methods until they support the standards, especially given the silent fail.

I don't mean to sound a bum note. The app looks great and I like the features on offer (I've bought a licence). My bad luck was to start my first session by just importing standard data or not...

If Aeon is XML based perhaps consider an XML or OPML import if that's easier.

Anyway, congrats on launching and best wishes the future.

ma
matt
Posts: 1180
Joined: Mon Jul 30, 2007 9:35 am

Sat Jun 02, 2012 12:41 am Post

mwra wrote:CSV fails (silently) if - as is normal with CSV file - you include the target field names as line #1. I'd suggest the import filter ignore line #1 if it doesn't stat with a date. It isn't necessary to include the time element of the data. CSV import fails if any all values are quote-enclosed; this is an error - if nothing else if is the way a CSV field is used to delimit a value contain a comment. Currently you can't import a title like 'Hello, world' as it contains a comma.

Tab-delim import is ... broken. As with CSV it fails completely if the filed names are included in line #1. That said, without a standard line #1 all I ever get for the event is Start Date and nothing else if End Date isn't supplied. If it is supplied it is imported as the event title! I couldn't test the other fields.

My polite suggestion would be to disable the import methods until they support the standards, especially given the silent fail.

I don't mean to sound a bum note. The app looks great and I like the features on offer (I've bought a licence). My bad luck was to start my first session by just importing standard data or not...

If Aeon is XML based perhaps consider an XML or OPML import if that's easier.

Anyway, congrats on launching and best wishes the future.


Hi,
Thanks for the comments - I have no problem with someone pointing out things that need improving, first day of launch or not!

I have fixed the import to handle header lines and quotes, and it will be available when I next put out an update, probably within the next couple of weeks.

If you are able to, it might be useful if you could email the file you are trying to import (if it does not have sensitive data) to support (AT) scribblecode (DOT) com, so that I can test that it works correctly with the improved importer I have just implemented.

Thanks,
Matt

mw
mwra
Posts: 21
Joined: Thu Jun 24, 2010 1:11 pm
Platform: Mac
Location: Portsmouth, UK

Sat Jun 02, 2012 6:58 am Post

Sure, no problem. Thanks for the quick follow up.

Ur
Ursula
Posts: 107
Joined: Mon Sep 18, 2006 7:52 am

Sat Jun 02, 2012 10:06 am Post

Does all this mean for once I'm not the idiot because I couldn't find a way to import a csv or tab delimited textfile created by Reunion to Aeon?

ma
matt
Posts: 1180
Joined: Mon Jul 30, 2007 9:35 am

Sat Jun 02, 2012 10:25 am Post

Hi Ursula,
The current version is able to import some CSV files, but not all.

If the file doesn't have any quotes around fields, or commas in place of titles etc. then it should be able to import.

Again, if you would like, you could send the CSV file through and I can look at why it won't work in the current one, and make sure it does work in the new one.

Matt