Computer vision approach to determine sleep disorders
Melih Gunay,a Mehmet Karakoc,bSelda Sarikaya Gunay,cCenkEvrend
aAkdeniz University—Department of Computer Engineering, Dumlupınar Boulevard, Antalya 07058, TURKEY
bAkdeniz University—Computer Sciences Research and Application Center, Dumlupınar Boulevard, Antalya 07058, TURKEY
cKemer State Hospital, Antalya, TURKEY
dMedlife Beylikdüzü Hospital, Istanbul, TURKEY
Abstract.If your doctor determines that you have symptoms suggestive of sleep apnea, then you may be asked to take a sleep evaluation test with a specialist often involving an overnight sleep. During the sleep, a variety of body functions, such as the electrical activity of the brain, eye movements, muscle activity, heart rate, breathing patterns, air flow, and blood oxygen levels are all recorded through polysomnogram (PSG). During PSG surface electrodes will be put on person’s face and scalp and will send recorded electrical signals to the measurement equipment. Among signals electroencephalogram (EEG) measures and records brain wave activity, electromyogram (EMG) records muscle activity such as face twitches, teeth grinding, and leg movements while also helping determine the presence of rapid eye movement stage sleep. In addition, belts are placed around the chest and measure breathing. While nasal airflow sensors record airflow and a microphone may record snoring activity. Even though PSG collects various data, because it is invasive the test results obtained are biased. Therefore in this study, we present a non-invasive image analysis based wireless sleep monitoring framework that is designed for easy clinical adaptation.
Keywords:computer vision, sleep disorders, image analysis, wireless sleep monitoring.
1 Introduction
Research has shown that sleep is one of the most important items for sustaining a healthy lifestyle. If we toss and turn continually throughout the night we never sustain a restorative deep sleep that our bodies and minds require. We wake after hours of sleep, but we hardly feel rested and rejuvenated (Horne, 2013; Kryger et al.,1994). Sleep can be divided into two main phases: (1) Rapid Eye Movement (REM) sleep and (2) Non-REM (NREM) sleep. NREM sleep has four stages of increasing depth from the Stage 1 (lightest) to the Stage 4 (deepest). During an ordinary night, if sleep does not cycle through these stages and repeatedly interrupted, sleep disorders hence side effects are experienced.
Sleep disorders can be divided into three major branches: 1) circadian rhythm disorders, 2) insomnia, and 3) snoring. Circadian rhythm disorders are often caused by the mismatch between the internal (biological) clock and the natural sleep. Circadian rhythm disorders include jet lag, adjustments to shift work and delayed sleep phase syndrome. Insomnia is a common problem characterized by trouble falling asleep, staying asleep or getting restful sleep, despite the opportunity for adequate sleep. Cognitive behavioral therapy for insomnia, often called CBT-I, is an effective treatment for chronic sleep problems. To make effective treatment, it is important to understand sleep cycles through detailed sleep diary for one to two weeks. Depending on the patient needs, sleep therapist may recommend a cocktail of CBT-I techniques. Snoring produces noise when the air person inhale rattles over the relaxed tissues of the throat. Snoring can be a problem simply because of the noise it causes. It may also be a marker of a more serious sleep problem called sleep apnea. Sleep apnea is a potentially serious sleep disorder in which breathing repeatedly stops and starts. You may have sleep apnea if you snore loudly and you feel tired even after a full night’s sleep. There are two main types of sleep apnea: (1) obstructive sleep apnea, the more common form that occurs when throat muscles relax, and (2) central sleep apnea, which occurs when brain does not send proper signals to the muscles that control breathing.
Diagnosis of sleep disorders could either be relatively straightforward simply observing disruptions to the daily pattern or fairly complicated requiring take Home Sleep Test (HST) or taking an overnight sleep study called polysomnogram (PSG) at a clinique. HST can be done in the comfort of home; however it records fewer body functions than PSG, including airflow, breathing effort, blood oxygen levels and snoring to confirm a diagnosis of moderate to severe obstructive sleep apnea. It is not appropriate to be used as a screening tool for patients without symptoms. It is not used for patients with significant medical problems and patients who have other sleep disorders such as central sleep apnea, restless legs syndrome, circadian rhythm disorders, insomnia, parasomnias, or narcolepsy in addition to the suspected obstructive sleep apnea. PSG, on the other hand, may be necessary if doctor determines that the person has symptoms suggestive of sleep apnea.
Fig. 1. A typical PSG test conduct and recording is shown (Colon, 2012).
PSG is performed in a sleep laboratory under the direct supervision of a trained sleep specialistsuch as shown in Fig. 1. During the test, a variety of body functions, such as the electrical activity of the brain(deCharms et al., 2005), eye movements, muscle activity, heart rate, breathing patterns, air flow, and blood oxygen levels are all recorded at night during the sleep. During PSG surface electrodes will be put on person’s face and scalp and will send recorded electrical signals to the measurement equipment. Among signals electroencephalogram (EEG) measures and records brain wave activity, electromyogram (EMG) records muscle activity such as face twitches, teeth grinding, and leg movements while also helping determine the presence of REM stage sleep. In addition, belts are placed around the chest and measure breathing. While nasal airflow sensors record airflow and a microphone may record snoring activity. Even though PSG collects various data, because it is invasive the test results obtained are biased(Barlow& Durand, 2009). Therefore in this study, we present a non-invasive image analysis based wireless sleep monitoring framework that is designed for easy clinical adaptation.
2State of the Art
Great interest is currently invested in wireless computing used for remotely monitoring and reporting of users’ health vitals. Composed of various sensors, both wearable and implanted, the technology relies on wireless connectivity to transfer the collected data to an Internet-based data management service. Being able to remotely monitor patient’s condition allows patients to maintain lifestyles and comfort while offsetting hospital management costs. Such a setup would facilitate giving early warnings to patients with heart or neurological problems, and providing a heads up for nearby caregivers.
2.1 Sleep disorder monitoring
The proposed Sleep Disorder Monitoring (SDM) is the most effective diagnosis approach for sleep disorders as it enhances HST with certain tests of PSG through wireless monitoring system. Such a system comprises the following components: 1) a fitted Wireless Sensing Network (WSN, including computer vision); 2) a gateway between the WSN and the Internet; 3) a service for data transfer over the Internet to a central server; and 4) a service for data processing, providing interface and alerts to the caregivers. For a successful and reliable operation of the monitoring system, a harmonious set of resources and functionalities needs to be simultaneously engaged in all components. The human body is a challenging terrain for wireless communication, especially given power constraints (due to limitations on the amount of energy allowed to penetrate the human body). Accommodating various levels of harshness in surroundings in terms of temperature, dust, humidity, etc., dictates a resilience requirement to an expanded set of failure possibilities that includes partial or complete failure of WSN elements, and reduced levels of activity or accuracy undergone as batteries deplete.
Equally, SDM needs to be able to maintain high integrity for the data collected. This requirement stresses both the integrity of SDM elements and the computations of the collected data at the central server(Shiomi et al., 2011). Ensuring comfort means that a fitted WSN needs to sustain different levels of mobility. Although the SDM elements rely on each other to gather and process data, mobility may be temporarily or permanently detrimental to the network operation (Yang, 2014; Chaudhary& Waghmare, 2014).
Fig. 2. A typical sound recording for analysis.
At the same time, network elements rely on batteries that are small in size in order to maintain user comfort. Prolonging network lifetime can hence be extremely challenging, especially when dealing with highly variable data such as neurological signals and body images. In (Wang et al., 2014), the authors have dealt with a novel evolution of duty-cycling through recognizing the viability of multiple states between completely asleep and completely active and propose effective powering schema.
For a prototype system, we designed a platform on which a high resolution (5+ MPixel) typical digital camera is installed on the wall of a bedroom. Although we did not complete the development of the custom software of the Next Gen HST App, we aim at running it on a mobile platform. For both practical and cost reasons, we are using all possible sensors available on the phone. Among these sensors; rotation, motion, proximity, sound, light and image are easily available. In addition, it is also possible to add-on heart-rate, temperature, humidity, pressure sensors at will.
In this study, we primarily focus on light, sound, and vision. The data that is collected was transferred to a central server for processing and advanced analysis. Furthermore, the data is also made available to the caregiver through the web-based application developed.
2.2 Light sensing
Sensing light is fairly straightforward. Knowing when the light is on could be important in properly establishing the night timeline.
Fig. 3. Detection of sleeping person.
2.3 Voice sensing
In a few study, researchers measured the intensity of sound and associated it to air passage rate to objectively quantify snoring (Hoffstein et al., 1991; Abeyratne et al., 2005; Jennum et al., 1985). However, these values depend on the microphone, the amplifier, the A/D converter, and the distance between the patient and the microphone. Furthermore, the signal energy intensity may vary with time in the same patient. In our study, we rather focus on snores by the analysis of temporal features of the snoring sound and quantifying the number of snoring and the duration of each snoring event(See Fig. 2). Other events that contaminate the signal such as external voice and coughing are eliminated. Unlike the gold standard PSG, the results obtained through voice analysis are reported to be satisfactory (Hsu et al., 2005).
Fig. 4. Fitting of body fit line.
2.4 Motion monitoring
During the night, the system tracks person’s movement in the bed using a basic infrared capable camera. The images captured could either be stored or may be broadcast over the Internet. The first stage of body segmentation process is the isolation of the person’s boundary (see Fig. 3 for a sleeping person). There are several methods for boundary detection (Zhou et al., 2010; Viola& Jones, 2001). For proof of concept, we simply used canny edge detection algorithm. Once person of interest is identified as shown in Fig. 3, the software fits a single bend line is fitted using the least squares method (Bretscher, 1995) as marked on Fig. 4.
Fig. 5. Various sleep positions (original image and edge image).
The length of each arm and the angle between the arms are measured and used to characterize the person’s sleeping behavior throughout the night. Fig. 5 shows various sleep positions and determined boundaries using simple edge detection algorithm.
In the next steps of the project, we plan to attach wireless motion and heart-rate sensors to human body to collect breathing and heart rate signals. Once these are collected, all these signals will be used to analyze sleep disorders in clinical studies (Morik et al., 1999; Koch, 2014). We will be associating sleep positions with sound signals and determine.
3 CONCLUSIONS
Smart hospitals can be effective to reduce costs of health systems and to provide high quality of patient care (Sanchez et al., 2008; Röcker et al., 2014). For instance to diagnose sleep apnea, doctors use PSG which requires surface electrodes and belts to be placed on person’s face and scalp to measure EEG, EMG, breathing, and movements; while recording snoring through a microphone. Even though PSG collects various data, because it is invasive the test results obtained are biased. In this study, we present a non-invasive image analysis based wireless sleep monitoring framework that is portable and cost effective and designed to be used at the comfort of home.
References
Abeyratne, U. R., Wakwella, A. S., & Hukins, C. (2005). Pitch jump probability measures for the analysis of snoring sounds in apnea. Physiol Meas, 26(5), 779-798.
Barlow, D. H., & Durand, V. M.(2009). Abnormal psychology: An integrative approach. Belmont, CA: Wadsworth Cengage Learning.
Bretscher, O. (1995). Linear algebra with applications (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
Chaudhary, D., & Waghmare, L. M. (2014). Design challenges of wireless sensor networks and impact on healthcare applications. International Journal of Latest Research in Science and Technology, 3(2), 110-114.
Colon, J. (2012).What is a sleep study and sleep fragmentation - paradise sleep [Website]. Retrieved from
deCharms, R. C., Maeda, F., Glover, G. H., Ludlow, D., Pauly, J. M., Soneji, D., … Mackey, S. C. (2005). Control over brain activation and pain learned by using real-time functional MRI.Proceedings of the National Academy of Sciences of the United States of America, 102(51), 18626-18631.
Hoffstein, V., Mateika, J. H., & Mateika, S. (1991). Snoring and sleep architecture. Am Rev Respir Dis., 143(1), 92-96.
Horne, J. (2013). Why REM sleep? Clues beyond the laboratory in a more challenging world.Biological Psychology,92(2), 152-168.
Hsu, Y. L., Chen, M. C., Cheng, C. M., & Wu, C. H. (2005). Development of a portable device for home monitoring of snoring.Journal of the Biomedical Engineering - Applications, Basis & Communications,17(4), 176-180.
Jennum, P., Schultz-Larsen, K., & Wildschiødtz, G. (1985). Snoring as a medical risk factor. I. Relation to arterial blood pressure, weight and use of antihypertensive medicine. Sleep Res., 14, 170.
Koch, I. (2014). Analysis of multivariate and high-dimensional data. Cambridge University Press, New York.
Kryger, M. H., Roth, T., & Dement, W. C. (1994). Principles and practice of sleep medice (2nd ed.). Philadelphia, PA USA: W. B. Saunders Company.
Morik, K., Brockhausen, P., & Joachims, T. (1999). Combining statistical learning with a knowledge based approach-a case study in intensive care monitoring. ICML, 268-277.
Röcker, C., Ziefle, M., & Holzinger, A. (2014). From computer innovation to human integration: current trends and challenges for pervasive health technologies. In A.Holzinger, M.Ziefle, & C.Röcker (Eds.),Pervasive health(pp. 1-17). Springer, London.
Sanchez, D., Tentori, M., & Favela, J. (2008). Activity recognition for the smart hospital. IEEE Intell. Syst., 23(2), 50-57.
Shiomi, F. K., Pisa, I. T., & de Campos, C. J. R.(2011). Computerized analysis of snoring in sleep apnea syndrome. Braz. Journal of Otorhinolaryngology, 77(4), 488-498.
Viola, P., & Jones, M. J. (2001). Rapid object detection using a boosted cascade of simple features. IEEE CVPR,1, 511-518.
Wang, J., Cao, Z., Mao, X., & Liu, Y. (2014). Sleep in the Dins: Insomnia therapy forduty-cycled sensor networks. IEEE INFOCOM, 1186-1194.
Yang, G. Z. (2014). Body Sensor Networks, Springer-Verlag, London.
Zhou, Z., Du, E.Y., & Thomas, N. L. (2010). A comprehensive sclera image quality measure. 11th Int. Conf. Control, Automation, Robotics and Vision (ICARCV), 638-643.