The recent advancements in information technology and computer science have led to the development of computerized information systems in different areas. In healthcare, these systems, known as healthcare information systems, are used as supportive tools to facilitate clinical and administrative tasks [1]. These systems can help to minimize errors, reduce costs and improve the delivery of services [2]. Nursing information systems (NIS) are among the key components of healthcare information systems [3]; they can help nurses to use data more effectively while managing their nursing activities, allowing them to improve patient care and develop their knowledge [4]. Nurses use a NIS to develop care plans, perform interventions, document patient care, organize information, and assess the effects and quality of nursing services [5678].

The efficient and effective use of a NIS depends to a great extent on the appropriate designs of these systems and how well they take the expectations and needs of nurses into consideration [910]. A poorly designed NIS can complicate the daily routines of nurses, possibly leading to problems related to the acceptance and use of these systems [1112]. The proper design of the user interface is one of the crucial elements affecting user interactions with the NIS. These systems should also be easy to learn and use [5]. The format of the data display, the format of the navigation instructions, and the data entry and data retrieval sequences should be meaningful for nurses, and they should find the NIS relevant to their work [13]. In other words, interactive health information systems including nursing information systems should be designed considering their usability features [14]. Usability is inherently linked to the quality of an information system [15] and by definition affects the extent to which such a system can be efficiently and effectively used to achieve specific goals and to ensure user satisfaction. Usability is directly related to productivity, error rates, alert fatigue, and user satisfaction [1516]. With respect to a NIS, usability problems mainly arise due to the limited functionality of these systems and the poor fit between the systems and the nurses' professional needs. Rogers et al. [17], in a study conducted using a scenario-based usability test, indicated that the application of human factors such as recognition rather than recall regarding the functions of a NIS and a good match between the system and the real world can help to improve the designs of NISs with regard to nurses' needs. Several researchers have undertaken usability testing with regard to other health information systems. Georgsson and Staggers [18], in their usability evaluation of a system for diabetes, used measures from the International Organization for Standardization (ISO 9241-11), specifically the IsoMetric usability questionnaire. Finding correct options and remembering the steps to complete tasks were among the issues reported. Similarly, Hamborg et al. [19] evaluated the usability of hospital information systems using the IsoMetric questionnaire. The findings showed low consideration of human factors such as learnability of the system and its suitability for performing tasks. Although several researchers have evaluated the usability of healthcare information systems, relatively few have investigated the usability of nursing information systems. This study aims to evaluate the usability of a NIS from the perspective of nurses. In detail, the following study questions are defined: (1) To what extent, do nursing information systems meet the ISO 9241-11 usability criteria? (2) What are the recommendations for further improvement of these systems?

This was a cross-sectional study conducted in 2015. The settings of the study were four hospitals affiliated with three medical universities in Tehran (Iran) that used a NIS developed by four major companies. Sampling in these settings considered two inclusion criteria. The first of these was that there had to be at least one hospital for every company that has a system installed in more than one hospital. The second was based on the number of hospital beds in general hospitals (i.e., general hospitals with the most beds). Initially, five hospitals were selected (one hospital from university 1, one hospital from university 2, and three hospitals from university 3). However, at the time of data collection, one hospital from university 3 did not agree to take part in the study and was thus excluded from the study. The final sample included four hospitals with four different NISs. To maintain confidentiality with regard to the information collected, the identities of the vendors and hospitals are obscured here. Instead, the hospitals are denoted as hospitals A, B, C, and D. Similarly, the software packages used in the hospitals are referred to here as A, B, C, and D, respectively.

The target population consisted of nursing staff at each hospital. However, a limited number of nursing staff members had access to the systems and used them. As a result, no sampling method was used, and all of the nurses who used the systems were invited to take part in the study (n = 184) (Table 1).

The data were collected using a usability questionnaire known as IsoMetric, which is based on the International Standard ISO 9241, Part 11. This questionnaire utilizes a 5-point Likert scale with options ranging from predominantly disagree (score 1) to predominantly agree (score 5), along with a ‘no opinion’ option (score 0). The original questionnaire contains 75 statements categorized in seven main groups (criteria): ‘suitability for the task,’ ‘self-descriptiveness,’ ‘controllability,’ ‘conformity with user expectations,’ ‘error tolerance’ (sensitivity), ‘suitability for individualization,’ and ‘suitability for learning’ [20]. However, due to the nurses' workload, which was expected to influence the response rate, the number of questions was reduced to 35, and each section consisted of five questions. The minimum and maximum score for each section was between 0 and 25. The validity of the questionnaire was assessed through expert opinions. The reliability was confirmed using Cronbach's alpha (α = 0.91). The questionnaires were distributed to the nurses at the four hospitals through the systems.

Data analysis was performed using SPSS ver. 19.0 (IBM SPSS, Armonk, NY, YSA). To analyze the data, the mean values and standard deviations were calculated, after which the systems were compared according to the usability criteria.

The response rate was 64.6% (n = 119). Ninety-seven percent of the participants were female. With regard to academic degrees, 97% of the nurses had a bachelor's degree and 3% had their master's degrees. In terms of the participants' age, the 24–30 age group accounted for 32% of all nurses, and the lowest frequency (8.4%) was that of age group of 45 years and above. The average age of the participants was 34 years (SD 7). Work experience for the majority of nurses (49%) amounted to 1–10 years, while 11.7% had work experience of more than 20 years. The average amount of work experience was 10.3 years (SD 6.9). In addition, for the majority of the nurses, the amount of experience with a nursing information system was between 1 and 5 years. The average was 5.4 years (SD 2.3).

Research suggests that the design of a health information system must fully match users' practices and must support them as they undertake their tasks; otherwise, the system will not be accepted by users or its uptake could face difficulties [21]. Among the criteria related to ‘suitability for the task,’ disagreement was mainly found for the factors of ‘easy to find required commands’ and ‘finding necessary information on a given screen.’ This is in line with the findings of Rogers et al. [17], which showed that nurses are occasionally unable to complete a task due to navigation issues and difficulties in finding the required functions. Another study indicated that displaying user-required information on different screens affected the efficient use of the systems and ultimately user satisfaction [22]. Similarly, research suggests that information about a given task should be displayed in a specific part of a system [23]. In general, features related to the ‘suitability for the task’ factor appeared to be considered when designing the systems evaluated in this study. The provision of information considering user needs, progressing through short steps to accomplish tasks, ease of access to required commands, and the display of necessary information on a single screen will improve the satisfaction of the user with the system.

In relation to ‘self-description,’ most of the nurses agreed that the NISs were self-described systems. However, the mean score obtained indicates that adequate attention was not paid to self-descriptiveness to the extent that it should when designing the systems. Regarding the sub-criteria of self-descriptiveness, the nurses agreed mainly on the clarity of the terms used in the systems and in the ease of understanding the messages displayed. This finding is in good agreement with the findings of Rogers et al. [17], in which nurses were reportedly positive about the consistency and clarity of the terms used in all aspects of the nursing system. However, the main area of disagreement was related to the capability of the systems to provide general explanations and relevant examples; these features should be considered in further developments of nursing systems.

Another feature of usability, in this case ‘controllability,’ as the findings showed, was taken into account in the designed systems. However, in the systems investigated by Rogers et al. [17], functions such as saving frequent actions or using shortcuts were not often readily available. Research points out controllability as a key feature of a well-designed information system [24]. The capability of moving between screens, stopping running procedures when needed, and returning to the main menu from any screen could help to improve the controllability of systems. Such features in turn affect user satisfaction.

Similarly, the majority of nurses agreed that the NISs conformed to their expectations. However, they expected to have more flexibility in terms of using identical keys for similar functions in all aspects of the systems. Studies have demonstrated the significance of considering user expectations and the related impact on user satisfaction with information systems [25]. Research suggests that information systems which do not meet user expectations are at risk of failure [26]. Using designations consistently in all parts of the systems, displaying messages at the same screen location, and predicting the time required to perform a given task are among the approaches that could help to improve system conformity levels with regard to user expectations.

In relation to the capability of the nursing systems to prevent errors, i.e., ‘error tolerance,’ as the findings indicated, more attention should be paid to this aspect in further developments of NISs. This finding is supported by the findings of Viitanen et al. [16], in which nurses reported making errors when performing documentation tasks, noting that the failure protection system was insufficient. These findings suggest that an appropriate interface design and reduced complexity of the systems could improve the error prevention factor. Studies have also reported the importance of information systems which are capable of preventing errors [2728]. Providing users with functions such as recovery from an error situation, restoring a previous session and correcting errors with little effort could help to improve the ‘error tolerance’ capability of a nursing information system [29].

Among the criteria outlined in ISO 9241, the lowest mean score was seen for ‘suitability for individualization.’ Near 50% of the nurses did not find their NIS suitable for individualization, and they pointed out that they were unable to adjust the response time of the NIS according to their working speed. This aspect should therefore be addressed in further studies of nursing systems, as it could help users to make more efficient use of the systems and could facilitate their work routines. A key factor related to ‘suitability for individualization’ is consideration of users' levels of knowledge and skills [29]. Studies have reported that information systems should be designed considering the knowledge and experience of their users. In addition, the systems should display information in an appropriate format to meet users' needs [29].

With respect to the last feature of usability, i.e., ‘suitability for learning,’ the nursing information systems were found to be easy to learn by the nurses, and the highest mean score among the seven criteria outlined in ISO 9241 was related to this feature. Nurses did not require much time to learn how to use the systems. In addition, it was easy for them to relearn how to use the systems after a lengthy interruption. However, in a study by Viitanen et al. [16], nurses found nursing information systems difficult to learn and not intuitive to use. Knowing that suitability for learning can influence user satisfaction levels [30], the designs of these systems should allow users to navigate the systems conveniently and learn them with little effort.

In this study, a modified version of the IsoMetric questionnaire was used to make it simpler to complete by the respondents. Although the reliability of the modified questionnaire was confirmed (α = 0.91), the study could benefit from using the original questionnaire to provide a broader picture of NIS usability issues. Moreover, in the current study, only a quantitative method was used to evaluate the usability of the nursing information systems assessed here. Conducting a mixed-method evaluation study (quantitative and qualitative) is suggested for future researchers to gain better insight into the usability problems of these systems.

In conclusion, this study evaluated the usability of four nursing information systems using the metrics recommended in ISO 9241-11. The findings showed that there are areas in need of further improvement. Providing nurses with the capability of individualizing the systems according to their work routines could help them to use the systems more efficiently and effectively. In addition, the designs of these systems should enable nurses to find their own way with little effort when using them. Moreover, the systems should guide users on how to perform their tasks conveniently. Finally, designing nursing information systems according to usability principles and considering user requirements can help to improve the effective use of these systems.