Svein Ølnes
common indicator set for all public organisations, the Danish system was changed to domain specific indicators, then to self evaluations by the public organisations themselves, and for 2012 the evaluation is reduced to an accessibility screening based on W3C’ s WCAG recommodations( W3C 2008) and the main weight is put on a user survey( Digitaliseringsstyrelsen 2012).
The Norwegian system, however, has been fairly consistent with the overall approach remaining the same and with only modest changes of indicators from year to year, at least the years 2007 – 2011 which is the actual period for this paper’ s investigations. In that respective the Norwegian evaluations system resembles the EU benchmarking system where the evaluation of the 20 basic services, 12 public services for citizens and 8 public services targeting businesses, has been the norm the last 10 years( Cap Gemini et al. 2010).
This paper looks specifically at the Norwegian evaluations of municipality websites. The first research question is whether the quality of the website as measured by the benchmarking system varies with the size of the municipality and if so, which underlying factors determine the variation in quality. Of course, the sheer size of a municipality in terms of citizens cannot explain differences in website quality in itself, but the bigger the municipality the bigger resources it also has, most likely also for the work with the website and the digital services.
To be able to explain the observed differences in quality, data from a survey to the administrators of the evaluated websites is used as well as the results from a general user survey where the users were asked to give their opinion and satisfaction with different public services, both physical and digital provided.
In chapter 2 the quality issue is discussed as well as different heuristic models and different evaluation models. Results from the expert evaluations and the web administrator and user surveys are presented and discussed in chapter 3. On the basis of this discussion the more general problem of measuring quality is raised again in chapter 4 and the paper concludes with a suggestion of a broader approach to measuring quality of public websites. The work is thus based on an inductive approach where we start off with the experiences gained so far before moving to explanation of differences in score and ultimately raising the discussion on how to best measure quality of public websites.
2. Can we measure quality?
Few words have been more used and misused then quality. Public as well as private companies emphasize the importance of increased quality of services but very often they fail to define what they mean by quality. Some simple definitions can illustrate this( Dahlbom and Mathiassen 1993, Braa and Øgrim 1995):
• a system’ s capability to satisfy needs, expectations and requests
• the proportion between expected and experienced yield of a system
These definitions emphasize different aspects of quality. The first one looks at measuring the difference between what is specified and what is measured or registered through‘ objective’ criteria while the other is based on experienced properties, that is,‘ a subjective evaluation from the individual user concerned’.
ISO‐8402 of the ISO‐9000 standard is guidelines for quality management and quality assurance and uses this definition of quality( despite the number the ISO‐8402 is part of the ISO 9000 standard):“ Quality is defined as the total sum of properties a unit carries and that concerns its ability to satisfy explicitly expressed or implied needs”. The ISO‐9126 definition of quality for software products, which websites after all are, is along the same line:“ The totality of features and characteristics of a software product that bear on its ability to satisfy stated or implied needs”.
Although these definitions are used much and are relatively general, they are like most ISO‐standards directed to processes and evaluation and the quality assuring of these. In the quality assessment of public web sites in Norway, a similar definition has been used:“ The quality of web sites in this project is defined as that public information and services on the Internet must meet a predefined standard or level that can satisfy some central user needs”.
390