In countries with advanced and modern healthcare systems, hospital pharmacies and their pharmacists play important roles in patient care [1]. In recent years, with the advancement of health information technology (HIT), there has been increasing attention to the adoption of the pharmacy information system (PIS) to improve the efficiency and effectiveness of services [2]. For example, the PIS has key roles to play in reducing medication errors and improving patient safety [34].

The PIS is a system that supports the distribution and management of drugs, identifying the type of intervention, determining the amount of inventory, reporting and managing of costs, and improving the accessibility of information [5]. Furthermore, the PIS helps clinical decision-making by alerting users about clinically important drug-drug interactions, drug allergies [6], and drug doses [7]; it also evaluates patterns of drug use [8] as well as other possible side effects of drugs. The PIS may operate as a separate and individual system or as part of a hospital information system (HIS), paired with the Computerized Physician Order Entry (CPOE) system [7]. Accordingly, to ensure efficiency and effectiveness of these systems, evaluation of the PIS is extremely important; this system could ultimately influence the safety and quality of care [9].

To identify and eliminate technical problems of health care systems, improve the efficiency and effectiveness of services, and minimize costs, careful evaluation of those systems is needed. In other words, the evaluation of information systems, as a key stage in the information system development life cycle [1011] can help to assure the technical capacity of these systems, determine the effects of using the systems on users practices, and allow application of modifications as required [1213]. Therefore, this study aimed to evaluate PISs in hospitals affiliated with three universities of medical sciences in Tehran, Iran.

The study population included the PIS implemented in hospitals affiliated to one of the three medical universities in Tehran, Iran (Tehran, Shahid Beheshti, and Iran Universities of Medical Sciences and Health Services). In Iran, more than 80% of hospital services are provided by the public sector. Medical education is integrated with the health services, which have created universities of medical sciences and health services. Each province has at least one such university. Tehran, which is a huge province with more than 14 million inhabitants, is divided into three zones. The three abovementioned universities provide public health and treatment services to the citizens of Tehran.

Since almost all of the affiliated hospitals implement a PIS from one of the well-known Iranian vendors, the sampling took place by considering two inclusion criteria: (i) there was at least one general teaching hospital for every vendor in which its PIS software was installed and (ii) hospitals were selected based on the number of hospital beds (the hospitals with the highest bed numbers). If a selected hospital was unwilling to take part in the study, the next hospital with the highest bed number was substituted. Selected hospitals provided 24 hr/7 days a week pharmacy services to in- and out-patients. The evaluation tool was a checklist created by reviewing the relevant literature; it was composed of general and specific evaluation criteria related to PIS [34514151617]. There were sub-categories for both general and specific criteria. Categorizing of the criteria and the related aspects was performed according to similarity and relevance. The general evaluation criteria included those aspects that are general to all information systems, such as security, user friendliness, and integration with other systems. The specific criteria evaluated those aspects that were specifically important for PISs. These specific criteria were divided into five categories including prescriptions and medication order management, patient safety management, purchase and sale management, drug stock management, and reports management. The checklist was constructed with 'Yes' and 'No' options and additional space was provided for any possible applicable comments. The content validity of the checklist was examined by five experts in the field of medical informatics and by two pharmacists.

Five PIS supplied by five vendors were evaluated in five general-teaching hospitals. Hospitals are called A (716 beds), B (426 beds), C (139 beds), D (111 beds), and E (379 beds). Similarly, the five software programs used in the hospitals are described as A, B, C, D, and E, respectively.

The findings are presented in sections. Section A reports the findings related to the general criteria, and section B describes the findings based on the specific criteria used to evaluate the PIS.

According to the findings, in terms of the security of the PIS, the users did not use appropriate passwords (a combination of capital and small letters, or a mixture of letters and numbers) to log into the system. This issue is considered a potential risk as unauthorized users could more easily gain access to confidential information stored in the system. According to studies conducted in other countries, the security of information systems will improve if a password is composed of a combination of numbers and letters [1415]. In addition, in the current study, only two systems had the capability to restrict repeated attempts by unauthorized users to enter the system. Troiano [16] suggest that a powerful security system is one in which the system administrator can determine the activity limit of users on the system and their accessibility. This finding is parallel to that of the current study, in which the administrators of the systems, based on their organizational roles in all five systems, was able to define users' access levels.

In terms of integration of systems, none of the PIS were connected with the national drug data bank, though, based on the law, it is not mandatory to do so. On-line updates through the linkage with the national drug information system may result in better patient safety. For example, the database of the Federal Union of the German Association of Pharmacists, known as ABDA, and the drug database of the United States (First Databank) are two that are connected with the national drug information banks of their countries; their data are updated through these databases [1718]. These databases include such things as drug interactions, financial information, drug prices, and drug side effects and possible adverse reactions [19]. Therefore, the lack of connection between the pharmacy information systems and the Iranian National Drug Information Bank may result in an increased number of errors, a reduced level of patient safety, and a lack of precise information about drug prices. This issue is considered one of the weaknesses of the existing software systems.

In terms of 'prescriptions management' and 'patient safety management', more attention was apparently paid to prescription order entries and delivery methods than to factors such as drug dose and frequency control, cautions regarding drug usage, prevention of duplication of medication orders, evaluation of drug interactions, and reporting of adverse drug reactions and allergies. This finding indicates that in the design of these systems, the vendors did not consider prescription management or patient safety requirements. This finding is in contrast with the study conducted by Hines et al. [20], who reported that the PIS under study had features such as drug interaction and contraindication alerts. All drug allergies and contraindications can be added as features to the existing PISs. Furthermore, the integration of the HIS system with CPOE and clinical decision support systems can help to improve patient safety [21]. In fact, the CPOE system is a part of clinical information systems (CIS) that is connected with other information systems such as the PIS [1922]. When the HIS does not provide this feature, many interconnected sub-systems such as PIS would not benefit from the integration. The majority of PISs in Iran are only designed to meet the financial aspects; the clinical aspects that can result in reduction of medication errors have been neglected. This finding is in line with the results of a study conducted by Alkelya [23], in which the limited capability of the PIS in supporting clinical tasks was reported as a key problem of these systems.

In this study, requests to purchase medicines, medical devices, and medical consumer goods from pharmaceutical companies and their distributors in the studied hospitals were made via fax or by sale representatives when they were visiting the hospitals. In other words, none of the studied hospital pharmacies was electronically connected to the producers or their distributors. Therefore, orders of drugs, medical devices, and medical consumer goods were not recorded electronically. This is one of the weaknesses of the majority of pharmacy information systems in Iran. This finding is supported by Asadi et al. [24]'s study, in which it was found that only one of the pharmaceutical companies had semi-computerized communication with the studied hospital pharmacy for marketing and sales purposes. However, in some developed countries, the pharmacy information system communicates electronically with pharmaceutical companies or their distributors, which reduces purchase and deposit process errors [25].

According to these findings, the evaluated systems had the capability of displaying the minimum inventory for each drug, defining different warehouses, and controlling the entry and exit of drugs from the pharmacy. This is in line with El-Mahalli et al. [26]'s study, in which inventory and purchasing management were considered as core functions of PIS systems. For online checking of inventory and distribution of medicines and equipment, Mahoney et al. [27] state that pharmacy software systems need to be integrated with other hospital sub-systems. In the present study, one of the systems did not even have a 'purchase and sale management' function. In the hospital using the mentioned system, a separate system was in use along with the PIS; this separate system controlled the inventory and the expiration date of drugs, and this system had no linkage with the PIS. Therefore, it was impossible to compare minimum inventories for drugs and to alert staff when order limits were reached.

As these findings indicated, all of the studied hospital PISs were equipped with dynamic report builders and were capable of providing customized reports on such topics as the total number of drug deliveries, the financial earnings of the pharmacies, the pharmacies annual performance, and various reports as required by the system administrator. This finding is supported by Isfahani et al. [28]'s study, which reported that the PIS understudy were capable of producing the reports required. This function plays an important role in managerial decision-making. Furthermore, insurance companies increasingly ask pharmacies for various reports, and their information needs are constantly changing. Therefore, dynamic report creation and the ability to customize reports are key requirements of pharmacy systems.