Tel-AvivUniversity – ComputerScienceSchool
Workshop in Computer Science
Multivariate Modeling
Sound and Interactive Multi-Media
0368-3500-07, Autumn 2004-5
Prof. Nathan Intrator
By:
Reuven Aronashvili
Oded Ashkenazi
Table of Contents
What is the ECG Analyzer?
Theory and Algorithms
Temporal Features
Amplitude Features
Patch Description
ECG_Analyzer_Test.mxb
ECG_Signals.mxb
ECG_Analyzer.mxb
Pass.mxb
Reset.mxb
Reset_loop.mxb
Max_Amplitude_Var.mxp
Min_Amplitude_Var.mxb
MinMaxRange.mxb
Signal_HighFactor_Filter.mxb
Peak2Peak_Time.mxb
R_Peaks.mxb
S_Peaks.mxb
R&T_Peaks
T_Peaks
Normalize_ms.mxb
STD.mxb
Normalizer.mxb
Output.mxb
Conclusion
Appendix I:
Importing Matlab data into Max
Appendix II:
What is ECG ?
Reference
Special Thanks to:
What is the ECG Analyzer?
The ECG Analyzer is the second module of a three module project. The first module is responsible for obtaining an analog ECG signal and inserting it into Max/MSP.The second Module is responsible for analyzing the ECG signal in real time and extracting a few important parameters which then get transferred to the third module,which is responsible for changing music in such a way that reflects the change in the given parameters. The changes are on wav and midi files.
The idea behind this project is that a man (or a women) running on a treadmill will be connected to ECG electrodes and wearing headphones, listening to his favorite music (mp3 player, Ipod, Discman etc.) He will receive immediate feedback about his running through the music that he hears. For example, if he is running too fast, therefore raising his hart beat to a dangerous level, the music might get louder and high pitched. This will make it clear to the listener that something is wrong, and with a little training, we could distinguish the different kinds of change in the music and adjust the way we run accordingly, for a safer and more efficient exercise.
This module analyzes the incoming ECG signal. We are receiving an ECG signal into Max/MSP, and using its tools to analyze it in real time. We used Max/MSP to implement this idea, because it is the natural place to work on music filters and effects. In addition to that Max/MSP has some very useful tools in signal processing which help a lot when analyzing the ECG signal in real time. We output 8 parameters, which will be used by later modules of the project.
Theory and Algorithms
The ECG signal has some very distinguishing features we wish to analyze. We can split the features we analyzed into two categories: Temporal features and amplitude features.
For a more detailed description on the ECG wave see:"Appendix I: What is ECG ?"
Temporal Features
The first feature is the time between the R peaks. This will be called R to R time. Because the R peak is the highest peak of the signal this is the easiest feature to detect. A simple peak searching algorithm would have been most suitable here. We implemented it in Matlab and it showed us the required results:
But since we are working in Max/MSP the simple peak searching algorithm we wished to implement isn't possible. Max/MSP is a tool for analyzing data in real time, it doesn’t have the functionality to go over a signal and find its peaks. We therefore turned to look for some real-time peak finding algorithms.
The first one we tried was "Pan-Tomkins algorithm for QRS detection". This algorithms is a real-time QRS detection algorithm based on analysis of the slope, amplitude, and width of QRS complexes. We will not go into detail about this algorithm because it too didn’t work in Max/MSP.
We ended up using the simplest version of the peak searching algorithm converted to work in real-time. We looked at the top 13% of the signal and we got a nice signal with only R peaks. We then moved the whole signal back to zero and used Max/MSP edge~ object to find the exact time the signal changes from zero to no-zero. Since the peaks are very small (we make sure they are by configuring the cutoff percentage chosen earlier) it can be assumed the are symmetric, therefore measuring the time between two peaks is the same as measuring the time between two zero to no-zero changes in the modified signal. We measure the time using the timer object.
This algorithm is really quite simple, but it works really well in Max/MSP and it uses its build in objects in a the way they were supposed to be used. That is the main difference between this algorithm and the ones mentioned above, and this is why the results we got from it where the best.
To analyze the time between the S peaks we used the same general method with a few minor adjustments to detect a minimum peak instead of a maximum one.
Amplitude Features
The next few features we wished to detect were the variance in the peaks amplitude (both for the R S and T peaks). Since we now know when the peak arrives we can use it to open a window and capture the amplitude values. To do this we used the MinMax object. By sending a reset message before every signal (using the temporal peak detection object) we captured the relevant values in every cycle. We used this for both the R and S peaks.
To make sure noise and small errors will not increase with time and interfere with our experiment we send a reset message to every object, every 20 seconds. This was tested and found to make our parameters less noisy, and more accurate.
The next feature of the ECG we wished to analyze was related to the T wave. In order to calculate it's variance and the T-S value we needed to find the T peaks. This is a little more complicated then the R and S peaks because it is not the maximal or minimal value of the signal. It is however the second highest peak in the signal, we used this information to find it. Again we set a cutoff value of 75%. This would generate both R and T peaks. Now we used this signal to open a small time window and then capture the maximum of the ECG signal in that window.This will generate a wave of the maximum amplitude of the ECG in each window. For a single cycle of the ECG signal, the first maximum is R the second is T.Now we selected the minimum of this wave. We received the T peak's amplitude. We then send this information to the object mentioned above to calculate the T variance. And we subtracted the S value to calculate T-S.
Patch Description
We will describe the patches we used to implement the algorithms mentioned above. All patches where implemented and tested in Max/MSP 4.5.1
Each patch has its own .mxb file, and can be used by itself. Notice some patches use other patches and so on, therefore if you wish to use some of these patches, make sure you have all sub-patches too. All of our patches are based on Max or MSP built in objects, or on other patches we created.
ECG_Analyzer_Test.mxb
This is the interface for testing our patch:
Instructions:
Before you can begin using our patch you must have all our files in one directory, and if you wish to use the digitizer patch you should install it first. (See its documentation for more details.)
- Select ECG input source. (Left for digitizer / right for raw input file).
- Turn PatchON. (Light should turn red).
- If input is a raw file, click Load File and select a raw float64 file.
- Press PLAY. (Light should turn green).
- Wait 60 seconds until results start coming.
ECG_Signals.mxb
We use this patch to test our patch on recorded ECG signals.
Input:
The left inlet receives a bang when yhou want to load a file. It opens a "File Open" dialog box. You should make sure to select a raw data file, encoded in the float64 format.
The middle inlet receives a bang when you want to start playing the file you loaded. (Therefore the load file must happen before you do this)
The right inlet receives a bang when you want to stop playing the file.
Output:
The output is an MSP signal which is the streaming of the raw data according to the specified sample rate in the DSP Options window. We used a sampling rate of 16000 kHZ in order for the digitizer~ to work. (see sfplay~ and digitizer~ documentation for more info).
Comment:
We used the Float64 format because it was convenient for us to transfer raw data from Matlab to this format, and the precision is the most accurate. Other encoding formats can be used – just change the 'samptype' argument in the code.
See: appendix I - "Importing Matlab data into Max", for more details.
ECG_Analyzer.mxb
This is the main patch of our project:
Input:
The first inlet is for an ECG signal (in the form of an MSP line).
The second input receives a bang, to start analyzing the signal.
The third input receives a bang to stop analyzing the signal.
Output:
We output 8 parameters in the form of a max float value. The output parameters are updated every 5 seconds. The actual value is the running average of the values of the 30 seconds. We wait an additional 30 seconds for the signal’s std and mean. (Therefore we will not output any result for the first minute after receiving the play bang)
In order to minimize errors generated by noise we send a reset signal every 20 seconds. This resets the amount of cumulative errors generated throughout the algorithm and causes the output parameters to be more accurate.
There are two types of parameters:
- Time intervals – These are positive parameters, and are given in 2 formats – milliseconds and normalized values with a mean of 0.5. (see: Normalize_ms patch ).
- Amplitude parameters – normalized with the original signal’s std.
The parameters from left to right are:
- R to R time in milliseconds.
- R to R time normalized.
- Variance of the R peaks amplitude.
- S to S time in milliseconds.
- S to S time normalized.
- Variance of the S peaks amplitude.
- Variance of the T peaks amplitude.
- T – S amplitude values.
Pass.mxb
We use this patch to split our signal into two different ranges: the left is 0-2 and the right is -1 – 1.
This is done to simplify calculations for temporal and amplitude features. For temporal features the 0-2 range is used, and for the amplitude features the -1- 1 range is used.
Input:
ECG signal with range 0-2.
Output:
The left outlet is the incoming signal.
The right outlet is the signal minus its average, resulting in range of -1 – 1.
Reset.mxb
The reset patch is used to send two reset messages, one immediately, and the other with a delay of 3 seconds.
This is used reset our min max counters and reset previous output values.
Input:
A bang.
Output:
The left outlet sends the reset immediately, the right with a delay of 3 seconds.
Reset_loop.mxb
The reset loop patch is used for resetting maximum and minimum values accumulated over 20 seconds. This reduces the effect of noise and spikes in the data.
Input:
1 - start the reset loop. 0 - stop it..
Output:
We output a reset and a bang together every 20 seconds.
Max_Amplitude_Var.mxp
Used for calculating the maximum amplitude's variance.
Input:
The left most inlet is the input signal. (created for ECG signals)
The middle inlet is a reset line for updating the signal’s amplitude on each cycle.
The right most inlet is a delayed reset line for ignoring the noises of the signal and trying to minimized their affect on the results. We will send a reset message every 20 seconds.
Output:
A signal that holds the maximum amplitude’s range.
Min_Amplitude_Var.mxb
Used for calculating the minimum amplitude's variance
Input:
The left most inlet is the input signal. (created for ECG signals)
The middle inlet is a reset line for updating the signal’s minimum on each cycle.
The right most inlet is a delayed reset line for ignoring the noises of the signal and trying to minimized their affect on the results. We will send a reset message every 20 seconds.
Output:
The left outlet is a signal that holds the minimum amplitude’s range.
The right outlet is the S peak’s amplitude value signal.
MinMaxRange.mxb
We use this object by sending a signal of the minimum or maximum amplitude of the wave. We then receive the range of that amplitude.
Input:
The inlet receives both the signal and a reset line. A reset message opens a time frame, where we will calculate the maximum and minimum, until the next reset.
Output:
The difference between the maximum and minimum amplitudes of the signal since the last reset.
Signal_HighFactor_Filter.mxb
Used in each patch that analyzes positive peaks and amplitudes data, e.g. R peaks & T peaks.
Input:
The leftmost inlet is the signal, which supposes to be filtered by this device.
The middle inlet is the reset line, which gets a reset every time we want to update the current maximum (in our case in every cycle).
The rightmost inlet is the factor, the exact percentage, which we want to cut, e.g. a factor of 0.81 will leave only the top 19% of the original signal (starting from zero).
Output:
The filtered signal. The output is the upper percentage, which we’ve chosen by the factor, reduced to zero. (see figure)
Peak2Peak_Time.mxb
We use this to measure the time in milliseconds between two peaks and supply resets before and after peaks.Used, for example, to open a window for capturing the T wave.
Input:
An MSP signal. (works best on signals with small amplitude peaks. see examples)
Output:
The left most outlet is the time interval between two peaks.
The middle outlet is sending a reset message when the signal changes from zero to non zero.
The right most outlet is sending a reset message when the signal changes from non zero to zero.
R_Peaks.mxb
We use this patch to generate the R peaks of the ECG signal. We used the factor 87% which will leave us with the upper 13% of the signal, reduced to zero.
Input:
An MSP signal.
Output:
The upper 13% of the input signal, reduced to 0.
S_Peaks.mxb
Input:
The left inlet is an ECG signal.
The right inlet is a reset line. It gets a reset massage every cycle in order to find the new minimum in every cycle.
Output:
The outlet is the lowest 13% of the input signal reduced to zero (negative peaks).
R&T_Peaks
This patch is used to send (later) reset messages to the T_Peaks_Only patch. This patch will be the input for the P2P_Time patch, and as a result we will open a time window for T peaks and R peaks one in a time. It will help us getting a signal holding only R values and T values.
To implement this patch we are using the Signal_HighFactor_Filter with the cut factor of 0.75.
Input:
The left inlet is the ECG original signal, which we are interested in its T peaks.
The right inlet is a reset line, which will be connected to the main reset patch. It is used to update the maximum in each cycle.
Output:
The upper 25% of the original signal reduced to zero. (see figure)
T_Peaks
Input:
The left inlet gets the original ECG signal and also a reset line. The reset line will be connected to the P2P_Time patch, which will be connected to the R&T_Peaks patch, and will get a reset massage after R peaks and also after T peaks. (See algorithm description)
The right inlet is a reset line. It will get a reset massage after a cycle is completed in order to save the T peak of that cycle.
Output:
A signal holding the T values of the ECG signal.
Normalize_ms.mxb
This is used to normalize our temporal parameters, R-R and S-S times.
The signal is divided by twice its mean over the last 30 seconds.
Input:
Signal values in milliseconds.
Output:
Signal normalized with mean of 0.5.
STD.mxb
We use this to calculate the Standard deviation of the signal over the last 30 seconds.
Input:
The original signal.
Output:
The standard deviation of the signal .
Normalizer.mxb
We use this patch to normalize the output signals. We normalize by dividing the output signal by the ECG wave's std. (The mean of the ECG wave is 0 ).
Input:
An MSP signal of the outputted parameters.
Output:
The input signal normalized.
Output.mxb
We use this to average the results before we output them to the screen. We also wait 60 seconds before outputting the results. (30 seconds for the signal's mean, and 30 more to average before output.)