PDF
Citation
Print
I. Introduction
Taking prescribed pills correctly accounts for a large proportion of healthcare at home. However, busy modern people sometimes forget information about prescribed pills and how to take them accurately. In addition, people sometimes forget to carry or take pills, and if this situation occurs with diabetes or blood pressure pills, a dangerous situation for one’s health may rapidly ensure [1–3]. Therefore, a pill-taking aid tool is vital in families without medical personnel. Interest in helping people who need protection, such as the elderly, children, and others, has also increased significantly in the setting of healthcare services [4]. Active research and development on tools to aid pill-taking are being conducted to meet this demand [5,6].
Since the easiest way to identify a pill and obtain information about it is to distinguish it visually, many studies on image processing technology have used images of pills [7–9]. Researchers have also recently used deep learning technology to improve the performance of pill-taking aid tools [10–12]. Deep learning models require large quantities of data in the learning stage; therefore, the demand for pill image data has also increased significantly.
In related works, Chang et al. [10] developed a wearable smart glasses-based drug pill recognition system using deep learning for visually impaired chronic disease patients. Their system consists of steps in which a patient puts a pill on his or her palm ad takes a picture; the system then classifies the pill through deep learning and guides the patient on how to take it. Wang et al. [13] studied recognition using minimally labeled data. Their study used front-side and back-side images they obtained themselves, as well as a consumer data set provided by the NIH Pill Challenge. In another study, Zeng et al. [14] created a mobile deep learning system for recognizing unconstrained pill images. They performed data augmentation to address the lack of a large volume of training images.
Currently, no automatic system exists for generating pill image data, Those who need images of the pill shot it manually and use them. However, taking a picture of a pill on a flat floor only results in images of the front and back of the flat pill. Another problem can be that a pill without a sharp angle rolls around, making it challenging to take pictures of the pill in a specific orientation is challenging. The image data generated in this way have a limited ability to contain information on pill appearance. The low quality of the data may diminish the accuracy of experiments based on the pill images.
Therefore, we propose a system to automatically generate image data by taking pictures of pills from various angles. This system is referred to as the pill filming system in this paper.
II. Methods
We designed this system to have three components: the instrument structure, control part, and graphic user interface (GUI) to assist in the visualization of system operation. This system focused on capturing the appearance of pills from various angles, with the ultimate goal of making images that are easy for researchers to use.
1. Structure Design
Figure 1 shows that the DC motor, motor cap, and column involved in the device’s power are located at the bottom. The device converts the rotary motion of the DC motor into a linear motion using a crank, which works as a seesaw moves. When the seesaw operates, columns next to each side of the seesaw fix the seesaw’s motion axis. A motor cap fixes the DC motor to the baseplate and reduces vibrations generated during device operation. A camera holder secures the position of the web camera to see inside the seesaw from above vertically. The camera guide adjusts the height of the camera holder from the baseplate along the groove on the side so that the web camera can locate and focus on the pill inside the seesaw.
Figure 2 shows the interior appearance of the seesaw, which has two bumps (referred to as “hills”) of different sizes inside, and these hills cause the pills to roll around and change direction and position while the seesaw operates. When the seesaw is in the direction where the stage is facing down, the pills stop on the stage in a random direction and position. At this time, the stage floor has a slope, so various configurations occur where different sides of the pill lean against the wall of the seesaw. This system was manufactured with black polylactic acid using a 3D printer for lightweight and easy manufacturing.
2. Controller Design
Figure 3 is a block diagram of the circuit configuration of the pill filming system, which consists of a power supply, mainboard [15], web camera, motor controller, and DC motor. The external power supply (5 V and 3 A) provides sufficient voltage and current for the operation of the mainboard and DC motor. The mainboard manages the pill filming system, controls the functions of the connected parts, and stores data.
We use the H-bridge motor driver L293D (STMicroelectronics, Geneva, Switzerland) as a motor controller, and it converts the 5 V voltage supplied from the mainboard into 12 V and delivers it to the DC motor. The DC motor’s speed is 15,000 rpm and the gear reduction ratio is 150:1; consequently, the number of rotations is 100 rpm. The motor speed is controlled by setting the duty cycle to 13%, thereby obtaining approximately 14 images per minute.
While the DC motor is running, the seesaw moves and the pill inside the seesaw has a random direction and position on the stage. When the seesaw finishes its reciprocating motion once, the web camera automatically focuses on the pill and captures the screen projected by the lens. The mainboard saves the generated images. After generating a total of 300 images, the system is shut down.
3. GUI Design
As shown in Figure 4, we designed the GUI to visualize system operation using Qt Designer (The Qt Company, Espoo, Finland). The word on each GUI button is a label on the pill surface, and we select it as a tool for identification. When the user clicks the GUI button, 300 images of the pill of the user’s choice are placed in a directory on the mainboard that matches the corresponding label.
III. Results
Figure 5 shows an actual view of the pill filming system. The pill filming system’s overall size is 195 mm × 189 mm × 206 mm, and it weighs 0.52 kg, not including a pill.
The use of the pill filming system proceeds in the following order. The user inputs the label of a targeted pill to collect data into the button of the GUI and places the pill on the seesaw. Next, if the user clicks the button with the selected pill label, the DC motor of the pill filming system runs, and the seesaw moves. The pill inside the seesaw rolls randomly. After the seesaw has completed one reciprocating motion, the pill is placed on the stage in a random position and direction. The top web camera focuses on the pill and captures the stage area. The mainboard creates a directory with the label in the internal storage and stores the generated image there. The pill filming system is initially set to complete after generating 300 images, giving enough time for the pill to change its position and orientation. Users can choose how much image data they need for a given study and how long it takes to collect those images.
Figure 6 presents four examples each of two types of generated data, where the target pill is located on the surface background of white gauze on the stage. The image size is 1280 pixels × 960 pixels, the horizontal and vertical resolutions are both 300 DPI (dot per inch) and the file extension is .jpg.
IV. Discussion
The easiest way to identify a pill and obtain information about it is to observe it visually, which requires pill image data. In recent years, research using deep learning technology has been actively conducted, requiring large quantities of data. However, there currently is no tool capable of automatically generating image data, including the appearance of a pill. Researchers who need pill image data have manually taken pictures of pills themselves, which is cumbersome and has limitations in that it is difficult to photograph pills from a specific angle.
Therefore, this paper proposes a system that can automatically generate pill image data from various angles. The pill observation data from various angles in a 3D field includes many configurations. Furthermore, the data collected in the same controlled environment has a uniform background, making it easy to process the images. Large amounts of high-quality data from the pill filming system can contribute to various studies using pill images. As an example of using pill images, if a neural network is trained with the data obtained from the pill filming system, it may be possible to develop a model that can determine the type of pill with only one picture. Abundant data facilitates high judgment accuracy of neural networks, so we initially set the pill filming system to generate 300 images [16]. However, the user can set the amount of data collected and the execution time through the GUI.
The pill filming system proposed in this study does not have a separate lighting system. The original image obtained from the pill filming system contains shadows. Therefore, in a following study, we intend to refine the pill filming system by including additional lighting devices such as a shadowless lamp used in a hospital’s operating room.
XML Download