Breathing Detection Using Computer vision with webcam
使用電腦視覺與網路攝影機進行呼吸偵測

Introduction  介紹

When I worked on the development of a healthcare device for detecting baby Apnea (pause of breathing), I have spent some time on studying possible technological solutions. I have tried several different types of sensors including: RGB-camera, 3-axis accelerometer, heat camera, depth camera (kinect), etc. The rgb-camera-based solution is very interesting. Firstly because they are low-cost; moreover, it does not require mounting the sensor on baby's body thus the comfort is guaranteed. That's why I worked for about 3 weeks on this solution.
當我在開發一種用於檢測嬰兒呼吸暫停的醫療設備時，我花了一些時間研究可能的技術解決方案。我嘗試了幾種不同類型的傳感器，包括：RGB 攝像頭、3 軸加速度計、熱攝像頭、深度攝像頭（kinect）等等。基於 RGB 攝像頭的解決方案非常有趣。首先，它們成本低廉；此外，它不需要將傳感器安裝在嬰兒身上，因此舒適性得到保證。這就是為什麼我花了大約 3 個星期在這個解決方案上工作。

As experiments, I tried multiple possible algorithms including: feature points tracking, inter-frame difference checking, flux, and color nuance variation tracking.
作為實驗，我嘗試了多種可能的算法，包括：特徵點追蹤、幀間差異檢查、通量和顏色細微變化追蹤。

In this post, I want to present the latter one. This work is inspired by this study. The basic idea is to track the color variation of an ROI on human face. The blood circulation and breathing will cause slight color variation of our skin. This change is so slight that our eyes can not perceive it at all. However, a normal webcam can identify color variation with a resolution as high as 2^16 grades.
在這篇文章中，我想介紹後者。這項工作受到這項研究的啟發。基本思想是追蹤人臉上感興趣區域的顏色變化。血液循環和呼吸會導致我們的皮膚顏色微微變化。這種變化非常微小，我們的眼睛根本無法察覺。然而，一個普通的網絡攝像頭可以以高達 2^16 級的解析度識別顏色變化。

ROI on human forehead  人類額頭上的投資回報率

I have chosen a small rectangle area as ROI (region of interest) for tracking color variation. Below is the part of code for this purpose.
我已經選擇了一個小矩形區域作為追蹤顏色變化的 ROI（感興趣區域）。以下是此目的的部分代碼。

x1 = 0.4
x2 = 0.6
y1 = 0.1
y2 = 0.25
face_cascade=cv2.CascadeClassifier(cv2.data.haarcascades + "haarcascade_frontalface_default.xml")

def getFaceROI(img):
    gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.2, 5)

    if len(faces)>0:
        img=cv2.rectangle(img, (faces[0][0] + int(x1*faces[0][2]), faces[0][1]+ int(y1*faces[0][3]) ), (faces[0][0]+ int(x2*faces[0][2]), faces[0][1]+ int(y2*faces[0][3]) ), (255, 0, 0), 2)

        return [faces[0][0]+int(x1*faces[0][2]),  faces[0][1]+ int(y1*faces[0][3]),  faces[0][0]+ int(x2*faces[0][2]), faces[0][1]+ int(y2*faces[0][3])]
    else:
        return [0,0,0,0]

Color Calculation  顏色計算

We simply calculate the average value of a given color channel (I used the green channel in my test) of all the pixels in the selected ROI.
我們只需計算所選 ROI 中所有像素的給定顏色通道的平均值（我在測試中使用了綠色通道）。

def getColorAverage(frame, color_id):
    return frame[:,:,color_id].sum()*1.0 / (frame.shape[0]*frame.shape[1])

Drawbacks  缺點

Entire code has been uploaded to github repo below:
整個代碼已上傳到以下的 github 存儲庫中：
https://github.com/QiuZhaopeng/CV2_heartbeat_breathing_detection

This solution suffers from limitations in multiple aspects, e.g. the lighting condition, human should stay still, the camera noisy level, etc. That's why I did not choose this solution for our product (finally we chose the 6-axis sensor solution).
這個解決方案在多個方面存在限制，例如照明條件、人體靜止要求、攝像頭噪音水平等等。這就是為什麼我沒有選擇這個解決方案作為我們的產品（最終我們選擇了 6 軸感應器解決方案）。

Result  結果

Below is the snapshot of the execution of this program. As one can see, there are low-frequency wave shape mixed with high-frequency sawtooth (small peaks).
以下是該程序執行的快照。正如大家所見，低頻波形與高頻鋸齒波（小峰）混合在一起。

I measured my pulses during execution of this program. It can be verified that each small peak corresponds to a pulse beat (heart beat). And the large peak-valley transition corresponds to breathing cycles.
我在執行這個程式期間測量了我的脈搏。可以驗證每個小峰對應一次脈搏（心跳）。而大的峰谷轉換則對應呼吸週期。

Anyone interested in this small experiment are encouraged to try it and do some further work. Good luck.
鼓勵任何對這個小實驗感興趣的人嘗試並進一步進行一些工作。祝好運。