13th European Conference on eGovernment – ECEG 2013 1 | Page 81

Juan Carlos Barahona and Andrey Elizondo

was used. If they did not agree, a third independent evaluator assessed the disputed variable and provided a third opinion. The third value was averaged with the closest of the original measurements.

The average review time was estimated at 45 – 60 minutes per site. This review was performed using a computer with manual and automatic metrics on the website. Institutions were not contacted at any time by the reviewer.

Once data was collected and processed, it was aggregated within the described framework and presented to public officials and the media through a national public innovation conference sponsored by the national e‐ Government authority. This diffusion process was part of the model, serving essentially three purposes:

• To raise awareness among public officials and leaders about the importance of committing themselves and their institutions to providing services by digital means

• To raise public awareness about the availability and quality improvement of public services offered by digital means and to stimulate demand

• To promote collaboration and competition by providing project or site managers with benchmarking information at a disaggregated level in order to guide and foster exchanges and support in areas with specific shortcomings.

This methodology was initially tested in Costa Rica 4 and adjusted between 2006 and 2009( Barahona and Zuleta 2006)( Barahona, Zuleta and Zamora 2008)( Barahona and Elizondo 2009). During the first trials, attention was mainly placed on the feasibility of capturing data; scaling the model 5 and having the country officially adopt the model and methodology. After three years of trials ‐and with little information collectedthe model was revised and adjusted by a panel of experts. Then, in 2010, the Government of Costa Rica adopted the model, and every year since then, an outside, multidisciplinary team from an international academic institution has systematically assessed government websites( INCAE 2010)( INCAE 2011)( INCAE 2012). In 2012, 132 subjects were assessed, representing more than two‐thirds of the country’ s public sector( estimated in terms of their aggregate budget compared to the national budget in the year 2011). After running the same model during the last three years, and with five times more institutions measured, we attempt to review some of the strengths and weaknesses of the data in order to suggest future directions for researchers and practitioners.

5. Data analysis and results

Table 2 presents the general results of the evaluation processes in 2010, 2011 and 2012; they suggest that of the factors evaluated, the weakest or lowest scoring variables were F1: Interaction, F2: Personalization and F6: Media The strongest or highest scoring ones were F3: Relevance, F4: Soundness and F5: Efficiency.

Table 2: General results for Costa Rica, by factor, 2010, 2011 and 2012 evaluations

2010( Obs = 108)

2011( Obs = 122)

2012( Obs = 132)

National progress or improvement on the Interaction, Personalization and Media factors will require commitment and leadership. The proposed index and methodology will allow public officials to focus their attention and resources on these areas. Both literature reviews and experience show that these three factors, due to their complexity, are those that require the greatest technological capacity and sophistication, elements that are necessary, but not sufficient, for making progress on public services offered by digital means. 4

Initially it was out of convenience, eventually the choice had to do with government support and funding to pursue a nationwide testing over time. 5 A total of 24, 34 and 104 institutions were included in for 2006, 2008, and 2009, respectively.

59