PRESERVING USER’S DATA
01/16/2017
Years ago, many scientists I knew used three Mac apps to plot their data, CricketGraph, KaleidaGraph, and DeltaGraph. Although some of these are still available, some after checkered histories of availability, most scientists I know have moved on to other options, especially R. These apps had proprietary binary data formats for storing the data, the scientist’s data. You were locked in, and when you could no longer use those apps, whether because you didn’t upgrade the license, or because you didn’t upgrade the software, or because your version no longer worked with your Mac OS version, you no longer had access to the data, your data. People were bitter, because returning to older projects or reanalyzing that data was impossible. Even Apple released a version of iWork that couldn’t read older versions, a decision that still seems astounding. For many people, but especially scientists, you depend on having access to your data, your work. It shouldn’t disappear with software.
In 2010, I launched one of my first commercial apps, Diversity, which is used to enter ecological data quickly, perform some common analyses, and export it into formats that other apps (like R) can easily use. Its approach to data entry not only made errors much less common, it made them easier to find.
A couple of weeks ago, I received one of those dreaded emails from Apple, that they were cleaning the App Store and that Diversity had not been updated recently. Apple told me that if I didn’t update Diversity within 30 days, it would be pulled from the App Store until I did. Fair enough, as I hadn’t updated it several years while I was deeply involved in iOS development. I was able to update Diversity quickly and keep it on the App Store. I changed little of the code, but realized that the app needed a major update, a project I’d pushed to the back burner for years.
I pushed it back for good reason. The code was Objective-C, not Swift. It used Cocoa, not UIKit, and I was rusty at Cocoa. It used xibs, not storyboards. It used bindings, which despite their benefits of saving boilerplate code, come at the cost of poor discoverability: there’s no easy way to track down all of your bindings. Worst, in 2010, I was green at object-oriented program, at Objective-C, at Cocoa, at Xcode, at version control. As a result, the project was a disaster zone. The app was even called DiversityAS, not Diversity, because I’d made a mistake when I uploaded the app to the store. Every time I opened the project, I groaned and closed it. Another day, maybe.
For the App Store update, I cleaned up some of the organization, but I knew the next step was to do the full storyboard and Swift conversion, along with improving the interface and the capabilities of the app. A decision loomed, though: this project could be an update of Diversity or it could be a new app to replace the aging Diversity, which would be pulled from the store.
If I went with the update of Diversity, I would have to preserve full backward compatibility with older data files. Unfortunately, I decided in the original Diversity to save the data as a binary file from Core Data. Although I did this to save file space (an important concern in 2010), it meant that the only way to read those files is with Diversity. If the new version could not read the old files, those files and their data would be lost.
My original plan was to follow this path and update Diversity. Given the need to convert from xibs to storyboards and from Objective-C to Swift, I decided based on migrating my other apps to not live in the interim hybrid world of both. I instead created a new project in which I would rewrite Diversity from the ground up. I would use the Objective-C files to guide my work, but I’d never have code that combined Objective-C and Swift. This was a far quicker and less error-prone approach than doing a piece-by-piece conversion of a hybrid project, and in about two weeks, I had a version that matched nearly all the capabilities of the original Diversity, along with a few new features.
Some of those new features involved changes to the Core Data model, and though I could accomplish this through lightweight migration, I could never get the new Swift version to read the data files made by the shipping App-Store version. I had thought it would be a simple procedure of importing the Core Data model from the old app, but it proved to be far more complicated. After fixing the obvious problems, the app simply threw an exception without any clues in the error messages about the nature of the problem. After multiple attempts, I realized reluctantly that my update wouldn’t be able to read old data files.
Although I could still ship it as an update to Diversity, the inability to read old files, to leave scientists stranded, made that solution unacceptable. Hearing the complaints from all those old plotting apps with their binary data formats, I knew I couldn’t go down that path. The app would be a new one, one that would replace Diversity.
Going this route also keeps those data files alive. Diversity has always had the ability to export all of its data as a comma-delimited table, and the new app can already import those files. Because I just updated Diversity on the app store, it will still be usable for several years, even if it is pulled from the App Store. I can let users know in the help pages for the new app and on the website that there’s an easy path for getting data out of Diversity and into Taxon.
The only drawback I see is that users paid for Diversity and now they would be asked to pay for Taxon. There’s some simple solutions to that, such as having Taxon available at a reduced price for an initial period. I’m also planning to add substantially more features, and that should help. This new appp will also avoid the binary trap by making xml the default file format, ensuring that users will always be able to read their data files.
In the end, preserving the user’s data matters most.