Hits up 35,307 in the last 7 days to1,661,336 hits from 100,222 unique users. Stats

AusGov.info - my mid year review

Article (1,730 words) posted on 17 May 2018 by Rosie Williams

AusGov.info went live in January in time for LinuxConfAu where, despite being so tired I could barely stand, I gave a potted history of my work with open data and the functionality of the site to a room full of enthusiastic developers. Reactions ranged from demanding why had I not achieved moar, moar, moar in the month since I decided to create the new project to a standing ovation from one audience member who, as part of team creating a similar (internal) project for the Victorian govt was incredibly impressed that I had done the work of an entire team single-handedly and in a month!

At this point I want to issue a very belated thank you to all the people I know personally, who turned up to hear me present at LinuxConf - the kindness of having so many people I recognised in the audience was not lost on me and I was very grateful for your presence :-)

I took the feedback I got from the audience, my wider Twitter network and people I had spoken to at other events and implemented site wide changes over the top of what I had built. I spent weeks designing the navigation using a complex interplay of CSS overlays and implemented charts that are generated on the fly that also act as navigation to allow people to drill down on the data by year for that data.

This replaced the top and bottom menu bars which forced people to click on a year whether or not there was any result for that particular year. With the new approach, people see from the chart whether there is a result in a particular year then click on that bar to see the breakdown for that year.

Since launch I have added the new political donations data in February, budget data in May and I also made the Taxation Expenditure Statement (tax concessions) data interactive. I worked so hard on the site that I didn't make a single blog post for about four months. Over the past few weeks since I stopped focusing on the code and turned my attention to providing journalism based on insights (made available through the AusGov) I've been somewhat confused about the level of support for the site.

While I have received a lot of positive comments and there is obviously decent traffic to both my blog and AusGov.info (below), I do feel that it is a struggle to get people engage the way I originally envisioned with AusGov as a research tool. I have come to suspect that people don't actually want access to large troves of data (to inform their own work) and I have set up a survey at the site itself to confirm or dispute this hypothesis.

I feel like I'm coming up against the limitations of relying solely on my Twitter network for exposure to my work but then with no effective income to spend on anything that I do, I'm not sure what additional options are open to me. I'm struggling daily against a sense of failure because despite the traffic, I'm not getting a sense that people are using the data in the site as intended. I have begun to suspect people prefer the blog posts to the data.

I am quite ok with the idea that people don't really want access to the database (and prefer the finished journalism to the raw materials) because trying to keep such an extremely large project bug-free is proving excessively stressful from day to day. So the idea of just running my own version offline and using it to inform my articles and politics would be a relief after so many months of shouldering the massive burden that the site so often feels to be.

It would also mean that I wouldn't have to try to design a site based on the needs of anyone and everyone and that can withstand demand from multiple users. If it is only me running my own offline version, that removes the limitations that come with creating queries that must run quickly enough to satisfy random users. I would also not have to deal with security in the same way if the site isn't publicly hosted.

Stats from my rosiewilliams.com.au (this blog) 2018

The site analytics confirm that it is the blog posts that get the bigger slice of traffic although when totalled, the hits overall are not that different between my blog and AusGov which is likely to be the effect of a much larger tail on the latter due to it having so many more combinations of results that a person (or bot) can hit on.

Stats from AusGov.info 2018 Stats from AusGov.info

While both sites captured over 100k hits for the year to date, (120,925 to the blog and 111,079 to AusGov) there are clearly more people visiting the blog and reading the posts than visiting AusGov. In April there was 6,279 unique visitors to my blog compared with 1,554 to AusGov and this trend is reflected for every month except the month AusGov launched.

This data stands in stark contrast to the financial support I receive for the work that I have completed. During the past months I have gained 5-6 new donors and lost probably the same number. I have received some additional donations through other means (which I am incredibly grateful for) but my stats on Patreon certainly don't reflect the enormous amount of work I have put in this year. It was particularly disappointing to do so much work over the month of May without receiving a single new donor for my trouble. There's not a lot of point in me putting myself through what it takes to keep such a big project online if at the end of the year I am still collecting in a week what a professional doing my job(s) would get in a single hour, every hour of every week.

Having said that, the decisions I'm making are not just based on money. I want to distinguish between people wanting AusGov online because they think other people will use it and whether there is anyone actually using the data in their own work (paid or otherwise). The question of whether to take AusGov down and just use it to inform my journalism is more about the stress it puts me under and where I need to be focusing my time.

To create blog posts and articles, I only need to ensure that the data I'm working with that day is all good (which I find quite manageable), whereas to keep a massive site with many different intersecting datasets and the jumble of code that drives the millions of potential results correct- as well as supporting other people to use the site- is a whole other ball game. So I'm very keen to understand just what it is people really want me to be doing with my time.

I think it is much easier for me to 'sell' my skills as a journalist than as a coder. Despite good feedback from the Paul Shetler's of the world, there are precious few coder types willing to give me credit (let alone actual donations) for the extraordinary level of labour and skill demonstrated in this project. So I have to question where I am putting my time and energy if the insights made available to AusGov.info are to survive in any form at all.

I want to be transparent to the few donors that I have about what they are getting in return for their donations. I also want to make sure that where I'm putting my labour and focus is actually what people really want (and not just what people think other people will use). I'll be looking carefully at the data over the coming month or so to try to deduce what it is telling me about what people want from me and when the time seems appropriate, make my decision.