Alexopoulos Charalampos et al.
The qualitative discussion can be based on the main evaluation dimensions and criteria of the PSI e‐ Infrastructures evaluation model described in the previous section, so it focus on the following topics / questions, presented in Figure 5:
Figure 5: Qualitative discussion
For the usability test, a number of potential users are asked to perform a scenario(= a series of tasks) using the PSI e‐infrastructure / platform under evaluation. There are three possible ways of practically conducting the usabilility testing:
• Peer testing by users: whilst one user is testing the platform and performing the tasks, the other user is monitoring and providing feedback to the session leader by filling in the evaluation form.
• Session leaders evaluate a random sample of users according to their choice.
• Individuals test and evaluate themselves( self‐organization principle): users perform the tasks and fill in the evaluation sheet themselves.
5. Application of the evaluation framework and results
In this section we report on the application of the proposed evaluation framework and procedure to the first version of the open public sector data e‐infrastructure developed in the ENGAGE project. In total, 21 students from Delft University of Technology and 33 students from the University of Aegean performed the student usability test by executing a series of tasks that require most of the available infrastructure functionalities, filled out the online user questionnaire and participated in a qualitative discussion. In addition, students of Delft University of Technology and the University of Aegean wrote reports about the usability of the ENGAGE e‐infrastructure for conducting open data download, process and upload scenarios. Five project members conducted the internal experts SWOT analysis. The results of these evaluation activities are as follows.
The application of the applied evaluation framework and procedure showed that the current ENGAGE e‐ infrastructure( ENGAGE 1.0) provides a good basic e‐infrastructure to start an open data project. The current e‐infrastructure enables searching, downloading, analyzing, uploading and rating data and also the manual linkage of data. Moreover, the ENGAGE e‐infrastructure provides a comprehensive overview of which open data e‐infrastructures currently exist in Europe. In addition, the ENGAGE infrastructure enables the usage of Application Programming Interface( API) for searching data, as well as, of a Wiki including tutorials. This means that the ENGAGE infrastructure manage to accomplish its technical objectives till the pilot evaluation.
Nevertheless, the e‐infrastructure could be improved in many ways. The application of the evaluation model and procedure showed that future versions of the ENGAGE e‐infrastructure should mainly focus on 1) difficulties in searching, need for more information about datasets, multilinguality, capabilities for combining datasets, and also for doing more complex analysis and performance, 2) more information about the quality of the data, for instance by providing a comprehensive rating systems for the data and 3) more information about
108