A few months ago, I was pointed towards a post that had some pretty big news on the government data front. The federal government would start posting summaries of completed Access to Information requests online.

I was looking forward to seeing how it would [...]]]> The Canadian Access To Information Database

A few months ago, I was pointed towards a post that had some pretty big news on the government data front. The federal government would start posting summaries of completed Access to Information requests online.

I was looking forward to seeing how it would be done – I do love the chance to sift through reams of government data, wild party animal that I am. So I was a touch disappointed when I got a look at the final result. The data was all there, but it was spread out over the different federal departments – all of them on their own sites, in their own formats. It wasn’t searchable, it wasn’t standardized and unless you knew exactly what you were looking for, you were unlikely to find anything of interest.

The discovery of ATI requests came at a time when I was also fine-tuning a web-scraping program that I was building, so I was on a hunt for some likely test subjects. I decided to use it to siphon off the data from a couple federal departments that I thought would have some interesting requests in them. At least, it started as “a couple”. Then “a few”. Up to “a handful” and then it was “a bunch”. Before I knew it, I had half the departments loaded in to my little database. And I figured “Why not the other half, then?”

Today, I’m posting the result of that “why not?” attitude: The Canadian Access To Information Database (CATID). It’s still very much a work in progress, but it contains all of the records that my web-scraper snatched from those federal sites (as well as a few stubborn entries that I had to grab with a very technical process known as “copy and paste.”) I figured that if I found it helpful to use, others might as well. The data is already out there, I just made it easier to look through.

I’ll continue to keep it as up-to-date as I can, as well as tweaking the site some. This did start as an experiment in web-scraping and then turned into an experiment in search-engine building, so I’m sure there will be some kinks to work out of it. If you do spot one, or just have a suggestion, scott@lilwall.ca is the place to let me know. Thanks!