Merging Five Years Of Data

In the end of 2010 I started tracking all of my nutrition, daily calorie burning, and moods full time.  Years of doing it off and on during 12-week fitness challenges, weight loss periods, et cetera prepared me well for this process and made it second nature.  Now it’s coming up on the end of 2016 and I have pretty much everything I’ve eaten, my daily calorie burn estimates, and my moods tracked for several years.  The problem is that the platform I had tracked it in, FitDay, is all but defunct.  How will I resurrect that data?

First the good news, FitDay had a PC program that I used for entering all my data.  I had relied on backing up to their website in case my hard disk crashed. About a year ago that whole transaction service stopped working and I was left for quite some time with my data trapped on the PC.  Countless months of inept service didn’t lead to any improvements on the matter so I started looking at options to build my own system to do the same thing and then figure out how to automatically export the data.  Meanwhile back in May I tripped across CRON-O-Meter  which did everything I had intended to and more.  After a few weeks of playing around I bought I lifetime subscription.  Now the problem became how to get all of that data merged between the two worlds.

My first thought was to write a program that would automate clicking the buttons in FitDay to get the data out.  I actually successfully got that working by building on an open source macro tool called AutoHotKey.  After some practicing I was sure I knew how I would extract at all of the data and even got a couple of weeks of trial extractions done.  Then it would be a simple matter of doing the same export from CRON-O-Meter, which actually has legitimate and easy offline extraction mechanisms, and merging into a local database.  At that point I started to get a little bit greedy and decided that wasn’t good enough.

FitDay has far better micronutrient tracking than most of the other fitness tracking systems.  They had a USDA database as their back end which had all of the macro- and micro-nutrients.  I also had a bunch of recipes built up from that same USDA catalog by making “recipes”.  So far so good, but CRON-O-Meter takes that to the next level.  They not only have all vitamins and minerals, like FitDay, they have a breakdown of the fat ratios, amino acids, and food acidity.  They track the water content of food, caffeine content of food, and can actually filter out supplements when doing reports.  They have several government databases, not just the USDA, and their databases are far more comprehensive.  Many of my custom foods under FitDay were commercial foods that I had to build because they never updated their database, while CRON-O-Meter has them as first class citizens.  If I just extracted from FitDay I would have a steep drop off in May 2016 in that granularity of data.

With all that considered I decided to do something far more manual, far more laborious, but far more ambitious.  I’m going to import all of my FitDay data, pretty much two thousand days worth of data, into CRON-O-Meter.  I’m going to do this manually.  Each day I’m going to enter my current day’s data like I always do plus 5-7 days of historical data.  By this time next year I will have over six years of high precision data covering all my nutrition and energy balance for pretty much every day (in the interest of disclosure I missed a few days back in 2011).  Hopefully the CRON-O-Meter team has anticipated someone keeping far more data than that in their account!