Alexopoulos Charalampos et al.
users a stronger role, beyond the passive consumption of content. This content increasingly includes commenting, rating and improving or adapting it to specialized needs, and then publishing it again, and also creating their own content. The clear distinction between data consumers and data producers does not exist anymore, leading to the emergence of‘ pro‐summers’, who produce and consume such data. These advanced PSI e‐infrastructures are also characterized by machine‐readable, linkable and context‐aware data and services for the use and the improvement and adaptation of open data by the end‐users( Papadakis 2012). Although the reuse of open data can be stimulated in different ways, e‐infrastructures are expected to play an important role in this direction by providing functions such as data processing, cleaning, curation and rating to the endusers( European Commission 2011). They are designed to allow communication from users( scientists, citizens, journalists, businesses etc.) to publishers( public bodies) and back( the feedback loop). Another characteristic of advanced PSI e‐Infrastructures is that in order to enhance interoperability they utilize richer metadata schemata.
4. Evaluation framework
An important objective was to propose a unified evaluation framework that can be used for the evaluation of both traditional PSI e‐infrastructures based on the web 1.0 paradigm and advanced ones based based on the web 2.0 paradigm. As it is described above, traditional and advanced PSI e‐infrastructures are both used by two main groups of stakeholders: the data providers and data users. In the case of traditional PSI infrastructures there are simpler functionalities corresponding either to the data users or to the data providers and these two groups of stakeholders have clearly divided roles. Consequently, diferrent capabilities are provided to each group, so two corresponding evaluation sub‐frameworks have to be built. On the other hand, advanced PSI e‐Infrastructures are characterized by frequent and repeated role switching when users interact with the platform. The proposed evaluation framework consists of an integrated evaluation model and a comprehensive evaluation procedure that utilises both qualitative and quantitative methods in order to get meaningful insights based on the evaluation variables and measures were identified during the construction of the evaluation model.
4.1 Evaluation model
For the above‐mentioned reasons we developed an evaluation model for the most complex and difficult case of advanced PSI e‐infrastructures 2.0, which is shown in figures 1 and 2. It consists of evaluation dimensions which are further elaborated into evaluation criteria, and covers both data users’ and data providers’ perspectives. So, if a user interacts with the system taking only the role of data user / provider, he / she will assess only value dimensions / criteria corresponding to data users / providers( denoted with‘ U’/‘ P’ respectively in the following Figures 1 & 2), while if he / she has both roles, he / she will have to assess all value dimensions / criteria. Similarly, if the proposed evaluation framework is used for the simpler case of evaluating a traditional PSI e‐infrastructures 1.0, it can be easily divided into two parts, one for the data users and one for the data providers.
It consists of the three evaluation levels proposed by( Loukis 2012) mentioned in section 2. Its first efficiency oriented level aims to assess the ease of use of the platform and also the usefulness of the basic capabilities it offers as proposed by TAM stream of research, covering both data and processing capabilities and also technical performance based on the information quality, system quality and service quality concepts proposed by the IS success stream of research. In particular, the capabilities assesed are: data provision, data search and download, data upload, data analysis and feedback, data curation. The second effectiveness oriented level aims to assess to what extent the platform supports the users to achieve their objectives and level of general satisfaction based on the‘ perceived usefulness’ concept proposed by both TAM and information success research, and the‘ satisfaction’ concept by information success research). Finally, the third future behaviour oriented level aims to assess‘ users’ intention to use’ the platform again in the future and recommend it to colleagues. The need for better presentation of the evaluation questionnaires and tools made us to merge the latter two levels in one, this of“ overall satisfaction”. Each of the above evaluation dimensions has been further elaborated into a number of relevant evaluation criteria.
105