Image processing can get pretty compute power consuming…

I wrote earlier a post on video surveillance using Python (see this post) – it works good and the neat thing is the Flask application that allows you to stream your video on a website. However I was not impressed by the speed of the video – it does not get smooth enough on a Raspberry Pi 3 – especially not if you start doing some more image processing… So I was wondering if things can get sped up using C++.
Here is an example with my cat detected:

Larger C++ projects can get hard to oversee. Fortunately you can take advantage of class structures and modularity of your library to have everything better organized.
Another challenge I always face is how to compile the entire thing in C++, especially when you have multiple modules. I found that in such a case cmake is a very helpful tool. The final result is an executable file.

Finally I want to show how you can set it up that the videos that are captured are automatically loaded on your cloud Dropbox folder, so that you can access them even with your smart-phone on a trip.

Motion Visualization with C++

There are several ways to get video streaming with C/C++, such as importing and using on Linux the native functions ioctl and video-for-Linux v4l (see e.g. the great post of JWH Smith).

OpenCV has some very useful routines to make the image/video capture in C++ pretty simple. This comes handy if you are also using openCV to do image processing with the captured images.

Required for this Project

This post builds on the openCV installation detailed in Webcam Video Access with OpenCV on Raspberry Pi. You will need OpenCV installed on your Raspberry Pi to be able to run the following code.
I was using as operating system (OS) the standard Raspberry Pi Raspbian environment. But cmake should make the entire code easy transportable to other systems.
The entire code can be downloaded from GitHub by typing at the command prompt:

$ git clone

The hardware I was using was consisting of:

  1. A Raspberry Pi 3 — This allowed a framerate of about 20fps (even though the code shows 30fps…) — with a 64GB SD card (such as this one). However 16GB would mostly be large enough.
  2. A Logitech webcam (such as this one).

…and the other usual periphery you need for the Raspberry Pi such as monitor (if you are not using remote login with SSH), keyboard and mouse.

The Structure of the C++ Library

In programming languages such as C/C++ you should always have a clear plan of the structure you need and the tasks your program should perform. In this [popup url=”” width=”600″ height=”800″]flowchart for main.cpp[/popup] you can see a flowchart that basically describes everything our main program does.

The file and folder structure for the C++ code might seem slightly complicated on first sight, but the modular build-up helps you stay organized once the code gets larger.

The files and folders for the here presented C++ routine are:


./include and ./detections are two folders. Here ‘time_stamped_video_files.avi’ are the output video files generated by the code when motion is detected. So at start the folder ./detections is actually empty (or has the README file from the GitHub repository).
The files in some more detail:

  • main.cpp: The main routine that creates instances of the class library.
  • include/camLib.hpp: The definition of the applied classes.
  • include/camLib.cpp: The member functions of the applied classes.
  • CMakeLists.txt: cmake library that enables simple compilation with openCV bindings.

The following chapters show the implementation and brief explanations of what is going on. Finally I will give you a description how the captured motion videos can be automatically pushed onto your Dropbox ‘cloud’ folder, which you can access from almost anywhere in the world where you are.

The Main Function

The main function main.cpp has the purpose to activate the camera and then in an endless loop to find out if anything moved in the field-of-view (FoV) of the camera. This all is achieved by calling member functions in ./include/camLib.cpp, which itself calls e.g. openCV functions.

A  main goal was to avoid writing video files on the SD card just to delete them once no motion was detected. This would reduce the life of the SD card. Better is to collect video frames in the memory (RAM) and only write the frames onto the SD card if motion was detected. If a sequence contained no motion the RAM is easy to erase.

Here a listing of main.cpp:

#include "./include/camLib.hpp"

int main(int argc, char* argv[])
  // Create a camera instance...
  Camera cam1;
  int number_of_changes = 0;
  const int frameBuffer = 50;	        // Frame buffer around motion ...
  const int lengthThreshold = 5;	// How many frames in a sequence need to show motion...
  // if more than 'there_is_motion' pixels are changed, we say there is motion...
  // and store the image...
  int there_is_motion = 5;
  int frameCounter  = 0;        // Counter for frames...
  int frameSequence = 0;	// Count sequence of frames...
  int framesTotal   = 0;	// Total number fa frames containing motion -- for saving...
  int saveVid       = 0;	// Flag to save keep video if changes found...
  vector*frameStack;	        // Buffer to store frames before saving to SD card...
  Mat frame;			// Captured single frames...
  int framesize;		// size of a frame...
  // Create file name for detected motion...
  time_t seconds;
  struct tm * timeinfo;
  char detectionfile[80];
  // Allocate memory for the frames to store and start the camera...
  frameStack = new vector [frameBuffer*sizeof(cam1.captureVideo())];		
  if(frameStack == NULL) {
	cerr << " Could not allocate enough memory..." << endl;
	return 0;
  // Start endless loop to capture frames...
  // This endless loop is stopped by user pressing the ESC key...
  // Generate new file name with a time-stamp right after the sequence that was captures
       // Display the resulting frames...
       frame = cam1.captureVideo();			// Capture a frame from the live stream of camera...
       framesize = sizeof(frame);
       // 'frameBuffer' times frames are the min amount of frames being captured per SEQUENCE...
       if(frameCounter < frameBuffer) { frameCounter++; // Count frames... // This line puts frame-by-frame on a stack...: frameStack->push_back(frame);	        // Put new frame on stack on the computer's RAM...
		// Only if there are more than 'there_is_motion' in the image
		// is it accepted that there is motion..
		number_of_changes = cam1.detectMotion();
		if(number_of_changes >= there_is_motion)
			cout << "...Motion Detected... number_of_changes= " << number_of_changes << " ... FramesTotal = " << framesTotal << endl; frameSequence++; // Need a minimum amount of frames in a sequence showing motion... // Was motion detected over multiple frames...? if(frameSequence > lengthThreshold)  {
				saveVid = 1;		// Set flag to keep this video, as motion was detected
				frameCounter = 0;	// Reset the frame-counter, so that more frames are captured after the motion detection...
		} else{					// If not enough motion detected, then reset the counter...
			// Reset...
			frameSequence = 0;
	else if(saveVid == 1)  {			// frameCounter >= frameBuffer and motion was detected (Flag 'saveVid' set)...
		// Reset the parameters for the next SEQUENCE...
		saveVid = 0;
		frameCounter = 0;
		frameSequence = 0;
		cout <<"......Save Video ....." << endl;			
		// Take frame-by-frame off the stack and save to avi video file on the permanent memory (e.g. SD-card)...
		cout << "Frames of the Video = " << ((frameStack->size())-1) << endl;
		// Get the current time...
		time (&seconds);
		timeinfo = localtime (&seconds);
		strftime(detectionfile, 80, "./detections/%d%h%Y_%H%M%S.avi", timeinfo);
		for(unsigned i=0; i<((frameStack->size())-1); i++){
			frame = frameStack->at(i);
		cout << "\n" << endl; // Release (close) the writer... cam1.closeVideo(); // Cleasr the stack, ready for the next SEQUENCE... frameStack->clear();
		framesTotal = 0;

	else  {						// In case no motion was detected, discard the images collected on the stack...
		frameCounter = 0;
		frameSequence = 0;
		saveVid = 0;
		cout <<"----- Nothing to save -------" << endl; frameStack->clear();
	if (waitKey(30) == 27) 		               //wait for 'esc' key press for 30ms. If 'esc' key is pressed, break loop
		cout << "esc key is pressed by user" << endl; frameStack->clear();

  return 0;

An instance of the Camera class is defined on line 6, called cam1.

On line 28 a memory stack named frameStack is defined which will collect the video images in RAM memory. Only if motion is detected will this stack be saved on SD card as a .avi video.

In line 37 an endless loop starts (ending on line 108). This endless loop is only terminated by pressing the Esc key, which is given in the code on line 90 to 94. Inside of this loop following steps are performed:

  1. Capture a frame (line 39)
  2. Collect a sequence of frames counting in frameCounter from 1 to frameBuffer (=50 on line 8).
    • If motion was detected (with the help of the function detectMotion()):
      • Was the motion observed in a continuous sequence of lengthThreshold (=5 on line 9) frames?
        • if yes then set the flag saveVideo to 1 (on line 51)
        • also set the frame counter frameCounter back to 0. This will make the captured sequence longer depending on for how long motion was detected.
      • If the continuous frame sequence was not long enough then reset the counter for the frame sequence frameSequence (on line 56).
  3. If finally frameCounter is larger than the value for framebuffer and additionally motion was detected in the sequence that was saved on the stack (indicated with saveVideo=1), then:
    • Reset flag and counters
    • Generate a file name with the date and time stamp
    • Open a video file with the generated file name (on line 73)
    • On lines 75 to 78 then are the frames from the stack taken on-by-one and saved in a .avi video file
  4. If frameCounter is larger than the value for framebuffer but no motion was detected (indicated with saveVideo=0), then:
    • Set all counters and the flag back to zero.

As mentioned earlier the [popup url=”” width=”600″ height=”800″]flowchart for main.cpp[/popup] can help get a better overview of the flow if the description here is not sufficient.

The Header File

The header file (here with the name camLib.hpp) defines the implemented classed and is basically a listing of all element functions and variables used in this class.
It also contains a listing of all required include files which are used in all other parts of the program.


#include "opencv2/highgui/highgui.hpp"
#include "opencv2/opencv.hpp"

using namespace cv;
using namespace std;

class Camera
	Mat captureVideo(void);
	Mat diffImg(Mat, Mat, Mat);
	int openVideoFile(char [80]);
	int saveVideo(Mat );
	int detectMotion(void);
	int closeVideo(void);
	VideoWriter writer;
	double dWidth;
	double dHeight;
	double fps;
	Mat frame;
	Mat prev_frame;
	Mat current_frame;
	Mat next_frame;
	Mat diff1, diff2;
	Mat motion;
	Mat result;
	int x_start, x_stop;
	int y_start, y_stop;
	int max_deviation;
	Scalar mean, stddev;
	int min_x, min_y;	// x, y coordinates of furthest left detected objects...
	int max_x, max_y;  	// x, y coordinates of furthest right detected objects...	
	int codec;
	int number_of_changes;
	Mat kernel_ero;		// Erode kernel...


The functions declared in lines 16 and 17 are the constructor and the destructor of this class, respectively. This is where the camera is activated and deactivated. The constructor prepares many things that are needed to start with the motion detection.
The other function in the public block of camLib.hpp are more in detail described below.

Important here is the definition of the VideoWriter object writer which is used below to write frames captured by the camera on the SD card.

The Member-Functions of the Camera Class

The functions declared in the header file are defined (implemented) in the member function file camLib.cpp. Here is the actual formulation of the functions.

#include "camLib.hpp" 

using namespace std;

VideoCapture cap(0);		// The openCV camera object...

// The Constructor...
Camera::Camera(void) {
	//Check if opening the camera worked...
        if (!cap.isOpened())  // if not success, exit program
            cout << "Cannot open the video cam" << endl;
	cout << " Camera warming up..." << endl;
	// Determine camera output size automatically...
	dWidth  = cap.get(CV_CAP_PROP_FRAME_WIDTH); 		// get the width of frames of the video
        dHeight = cap.get(CV_CAP_PROP_FRAME_HEIGHT); 	// get the height of frames of the video
        fps     = cap.get(CV_CAP_PROP_FPS);				// get frames-per-second of the video device
	// Print values out...
        cout << "Frame size : " << dWidth << " x " << dHeight << " --- fps: " << fps << endl; // Read three images first... cap >> prev_frame;
	cvtColor(prev_frame, prev_frame, CV_RGB2GRAY);
	cap >> current_frame;
	cvtColor(current_frame, current_frame, CV_RGB2GRAY);
	cap >> next_frame;
	cvtColor(next_frame, next_frame, CV_RGB2GRAY);
	// number_of_changes is the amount of changes in the resut matrix...
	number_of_changes = 0;
	// Detect motion in window...
	x_start = 10, x_stop = dWidth-10;
	y_start = 10, y_stop = dHeight-10;
	// Maximum deviation in the image, the  higher the value, the more motion is allowed...
	max_deviation = 20;
	// Erode kernel...
	kernel_ero = getStructuringElement(MORPH_RECT, Size(2,2));
	// Initialize Video Writer...
	codec = CV_FOURCC('H', '2', '6', '4');		//select desired codec (must be available at runtime...

// The Destructor...
Camera::~Camera(void)  {
	cout << "Shutting down camera and closing files..." << endl; 
        writer.release(); // Release VideoWriter() } 

// The camera access function... 
Mat Camera::captureVideo(void) { 
        // Read in a new frame... 
        cap >> frame;	                                // This frame is furthre processed for the motion detection...
	result = frame;		                        // This copy of the frame is used for the display in imshow...
	cvtColor(frame, frame, CV_RGB2GRAY);

        // Display frames...
	imshow("Motion Indicator", result);		// Display the current frame...
	// Put images in the right order...
	prev_frame	= current_frame;
	current_frame	= next_frame;
	next_frame	= frame;

        // Calculate differences between 3 consecutive frames...	
	motion = diffImg(prev_frame, current_frame, next_frame);
        return result;

// This function calculates the differences between the 3 consecutive images...
Mat Camera::diffImg(Mat prev, Mat current, Mat next)	{
	absdiff(prev, next, diff1);
	absdiff(next, current, diff2);
	bitwise_and(diff1, diff2, motion);
	threshold(motion, motion, 35, 255, CV_THRESH_BINARY);
	erode(motion, motion, kernel_ero);
	return motion;

// This function counts the differences between the 3 consecutive frames 
// and puts a rectangle around the area of motion...
int Camera::detectMotion(void)  {
	// Calculate the standard deviation...							
	meanStdDev(motion, mean, stddev);	
	// if not too many changes then the motion is real...
	if(stddev[0] < max_deviation)
		number_of_changes = 0;
		min_x = motion.cols, max_x=0;
		min_y = motion.rows, max_y=0;
		// loop over the image and detect changes...
		for(int j=y_start; j < y_stop; j+=2)  {		// height...
			for(int i=x_start; i < x_stop; i+=2)  {	// width...
				// Check if the pixel at (j, i) intensity is equal to 255 (= white)
				// this means that the pixel is different in the sequence
				// of images (prev_frame, current_frame, next_frame)
				if(static_cast(,i)) == 255)
					if(min_x>i) min_x = i;
					if(max_x<i) max_x = i; if(min_y>j) min_y = j;
					if(max_y<j) max_y = j; } } } if(number_of_changes) { //check if out of bounds... if(min_x-10 > 0) min_x -= 10;
			if(min_y-10 > 0) min_y -= 10;
			if(max_x+10 < frame.cols-1) max_x += 10;
			if(max_y+10 < frame.rows-1) max_y += 10;
			// draw rectangle around the changed pixel...
			Point x(min_x,min_y);
			Point y(max_x,max_y);
			Rect rect(x,y);
			imshow("Motion Indicator", result);
		return number_of_changes;	
	return 0;

// This function opens an avi video file, 
// using a codec that works with this file format (here: h264)...
int Camera::openVideoFile(char detectionfile[80]) {
	string filename = detectionfile;, codec, fps, frame.size(), true);
	//Check if succeeded
	if (!writer.isOpened())	{
		cerr << "Could not open the output video file for write\n";
	return 0;

// This function saves the frames in the avi video file...
int Camera::saveVideo(Mat oneFrame) {
	namedWindow("saving", WINDOW_AUTOSIZE);
	imshow("saving", oneFrame);

	//writer.write(oneFrame); //or...
	writer << oneFrame;
	return 0;

int Camera::closeVideo(void) {
	writer.release();	// Release VideoWriter()	
	return 0;

I will keep the documentation regarding the above file camLib.cpp only to the most essential points. More documentation is given a comments in the code.

camLib.cpp starts with the instantiation of the camera in line 5 as the global variable cap.

The class constructor in lines 9 to 44, which gets into action only once at the definition of a class instance, initializes and prepares everything:

  • Line 11 starts the camera.
  • Lines 22 to 27 capture 3 frames, which are used as initialization of the motion detection.
  • Line 43 defines the ‘codec’, which is required to write later the .avi video file on the SD card.

In lines 47 to 50 is the destructor of the class, which gets into action only once at the termination of the program and makes sure that all processes end properly.

The function captureVideo() in lines 54 to 72 captures a frame and calls the function diffImg() with the current and the two last images (called prev_frame, current_frame and next_frame).

The function diffImg() in lines 75 to 83 calculates (prev_frame – next_frame) and (next_frame – current_frame). These simple calculations leave everywhere ideally a 0 in the resulting two frames where nothing moved. But even if nothing moved unavoidable noise will result in not zero values even if nothing moved. An “AND” operation between the two resulting frames helps reduce false non-zero values.

The function detectMotion() in the lines 87 to 123. The openCV function meanStdDev() called in line 89 grabs the calculated motion image from line 69 and determines mean values and standard deviation in this image. With this we have a base-level that will help find ‘real’ motion.

This standard deviation is used as a ‘benchmark value’ in line 92, comparing it with a user-chosen limit-value max_deviation to decide if the camera detected motion. Motion is detected in the motion image from line 69 where the values are close or equal to 255. In lines 105 to 117 those areas are framed in by a yellow box to highlight where exactly the motion in the image is taking place.

The function openVideoFile() in lines 127 to 135 opens a new file.

The function saveVideo() in lines 139 to 148 uses the VideoWriter object writer to write frames on the SD card.

Linking and Compiling with CMAKE

Traditionally to compile and to link the modules or libraries of a C/C++ project together to get a functioning executable file, a Makefile is used. Writing a Makefile can get tricky, if a project is a little more complex.
CMAKE is a great tool to ease this pain somewhat. It is basically a tool that helps generate the more complicated Makefile, which then – using the command $ make links and compiles all libraries to one executable code.
The cmake I am using here to link and compile the above files is as follows and needs to be saved as CMakeLists.txt :

cmake_minimum_required(VERSION 2.8)
project( DisplayMotion )

# Add the path to the header file:

# GLOB allows for wildcard additions to include your code:
file(GLOB SOURCES "./include/*.cpp" "./main.cpp")

# Add the openCV parts:
find_package( OpenCV REQUIRED )

# Combine everything and give the executable the name 'displayMotion'
add_executable( displayMotion ${SOURCES} )
target_link_libraries( displayMotion ${OpenCV_LIBS} )

Detailled information on cmake can be found on the site.
The first two lines in this file define this file as a cmake script and specify the project name. The name of the final executable is defined in lines 14 and 15 as displayMotion.

Compile and Run

Finally we get to compile the whole thing and to run it…

As mentioned you need to first run cmake, which is done by typing and executing:

$ cmake .

(Don’t forget the ‘.’)
This generates a Makefile. Now comes the actual compilation with running

$ make

Now you can run the executable by typing:

$ ./displayMotion

A window should pop up on your screen showing a life stream fro your camera. When something moves in this video it should be framed by a yellow box.

Displaying the Saved Video

The video saved as .avi can be viewed on the Raspberry Pi with omxplayer.
You might have to install this first by typing:

$ sudo apt-get -y install omxplayer

To run the video simply type at the prompt:

$ omxplayer output.avi

Loading Videos onto DropBox

Saving the videos with detected motion on your SD card is already a good surveillance system. Better would be though if the videos are not only saved locally on your Raspberry Pi, but also saved in e.g. your DropBox folder, from where you could check for things happening from anywhere if you have internet access.
So first you need to have a DropBox account. If you don’t have that already, then please just follow this tutorial.
To setup your DropBox uploader on Raspberry Pi you can just follow [popup url=”” width=”600″ height=”800″]this great tutorial[/popup]. However the script from Andrea Fabrizi has become easier to use:
After logging into your DropBox account at, you will find further down on the page Generate access token with a button “Generate” below. After pressing that button you will get a really long random number/letter word. This word needs to be copied at the “Access token:” request when you run Andrea Fabrizi’s script as shown in below image:

Let’s say then that the DropBox folder you created is called “diningroom”. If you change now on Raspberry Pi into the folder where your motion videos are saved, you can upload all of them on your DropBox folder with following command:

$ ~/Dropbox-Uploader/ upload . diningroom

Certainly you want this upload to be done automatically. This can be done using Linux’s crontab. Open up crontab with the following:

$ crontab -e

In this editor add the following lines:

# Upload videos to dropbox:
/15 * * * * ~/Dropbox-Uploader/ upload ~/camClass/VideoCapture_version/detections/* diningroom; ls -t /camClass/VideoCapture_version/detections/* | tail -n +2 | xargs rm --
# Delete the saved videos at midnight:
1 0 * * * ~/Dropbox-Uploader/ delete diningroom
5 0 * * * ~/Dropbox-Uploader/ mkdir diningroom

Line 2 uploads every 15 minutes saved videos of the Raspberry Pi folder detections to the DropBox folder diningroom. Right after that all videos in the detections folder are deleted to make room for new videos and to avoid double-uploading onto DropBox. However care has to be taken to avoid that a video is being deleted while it is being saved – chances are not very high, but it can happen. In that case it would make the routine crash. The part after the semicolon in line 2 take care of that:

  1. List all files in the directory.
  2. Take all besides the latest in the list.
  3. Remove these (so the latest is not touched…).

One minute after midnight then (line 4) the diningroom folder on Dropbox is deleted with all saved videos in it. This is to free up space for new videos. So if you don’t save the videos somewhere else they will be gone at midnight everyday.
Line 5 then creates 5 minutes after midnight a new diningroom folder on DropBox ready for new videos with motion.

Set the Internal Time

You might also have to fix the internal Raspberry Pi time and time zone. I am using a Raspberry Pi 3 and fixed it with the help of this post.
Basically do the following:

  1. Stop the internal time:
    $ sudo /etc/init.d/ntp stop
  2. Set the right time-zone:
    $ sudo raspi-config

    …and choose ‘4 Internationalization Options’ to change the time-zone.

  3. Restart the internal time:
    $ sudo /etc/init.d/ntp start

Running in the Background

One more thing I would like to mention:
If you want to run the program for long hours (maybe day and night) you need to make sure that it does not get terminated when the Raspberry Pi goes in standby mode.
For this you can make the program run in the ‘background’ with following call:

$ nohup ./displayMotion &

This will start the program in the background and all console output will go into a file named ‘nohup.out’. The program should now no more be terminated when you close windows or leave your computer untouched for a longer time.

So this post got pretty long – but I hope still useful if you want to use C++ for some video surveillance…

…and as usual: I am looking forward to your comments and thoughts.
Please leave your feedback in below form…