José María Moreno‐Jiménez, Cristina Pérez‐Espés and Maria Wimmer
The first e‐participation experiences were aimed at analysing the extent to which the methodology( decisional tools) and the technologies( information and communication support) were user‐friendly, in other words, to determine if the products were efficient. Once the products’ efficiency was evaluated, the next step was to look at the efficacy of the processes. More specifically, the work focused on aspects such as citizen participation and control, transparency and accountability of the politicians. The third and final step in the evaluation of an e‐Government experience is the analysis of its effectiveness: the public valued added to society as a whole. This last question is very complex and there are almost no published works to date. The evaluation of effectiveness must be based on the mission of democratic models in every period of history.
Effectiveness( doing what is right) can be understood( García and Moreno‐Jiménez, 2008) as the identification of the aspects relevant to the problem and the setting of appropriate goals for resolving it. In this case, it involves the complete development of the capacity for working with a plural, coherent, hierarchical teleological system as a support to political action and this implies( García and Moreno‐Jiménez, 2008): a) the setting of aims of the highest order( subsistence, equity, liberty, knowledge, participation etc.) for the achievement of which, in the final analysis, the democratic system was designed; b) the identification of the relevant operational objectives associated with the aforementioned aims; c) the setting of precise levels( goals) for the objectives that must be achieved.
These three concepts( efficiency, efficacy and effectiveness) have been utilised when evaluating the cognitive democracy known as E‐cognocracy. This cognitive democracy uses multicriteria decision making techniques as its methodological support, the internet as its communication support and the democratic system as a catalyst for learning; it seeks the creation and social diffusion of knowledge and the construction of a more open, transparent, cultured, educated and freer society ‐ a society that is more cohesive and connected, more participative, egalitarian and cooperative. Structural equation models have been employed to analyse the relevance of the criteria for evaluating the effectiveness of e‐cognocracy( Mamaqui & Moreno‐Jiménez, 2009; Moreno‐Jiménez, Pérez‐Espés and Rivera, 2012a, b).
In Section 3, the ideas extracted during the evaluation of e‐cognocracy are used in the presentation of an original framework for the evaluation of the effectiveness of e‐governance experiences in the Knowledge Society, valid for any e‐participation experience in policy making.
3. A framework for effectiveness evaluation of e‐governance experiences in the KS
In line with the EF 3 ‐evaluation and based on the three inputs contemplated by the model( Delone & McLean, 2003), Moreno‐Jiménez, Pérez‐Espés and Rivera( 2012b) selected three relevant aspects as those that determine efficiency: i) the Information Technology application( System Quality), ii) the information that is obtained( Information Quality) and iii) the human resources support( Service Quality). There are four relevant aspects that determine efficacy: i) Information, ii) Communication, iii) Decision and iv) Participation Expectation. The evaluation of effectiveness is ideally undertaken by a group of experts that establish a‘ strategic plan’( an ideal situation) which is compared with the current reality. In this case, the evaluation from this perspective is through the analysis of two scenarios as latent intermediate variables: the current situation and the ideal, and an endogenous variable that captures the idea of the creation of a better society( Moreno‐ Jiménez et al., 2012b).
After identifying the relevant aspects for an EF 3 ‐evaluation of e‐Cognocracy, and with the aim of extending these ideas to any e‐participation in policy making, the next step in the design of a general methodology was to establish a framework with a set of criteria, subcriteria and indicators for evaluating the impact of e‐ governance experiences and collaborative democracy models in the context of the Knowledge Society and New Public Governance.
A set of attributes and indicators for evaluating efficiency and efficacy( as defined in this work) can be seen in Wimmer and Bicking( 2013), who propose a framework for the evaluation of the effectiveness( denominated as‘ efficacy’ in our context) of e‐participation solutions in engaging a wide audience in the policy making process( a“ crucial” factor, in the opinion of Macintosh and Whyte, 2008). For each of the selected attributes and indicators, different levels of impact( very high, high, medium, low, and very low) must be determined.
357