Seat Occupancy Detection

The system development of the project can be categorized into the following phases. 1) Capture image frames at intervals through a camera which does live monitoring of a room.
2) Interface camera to a single board computer for image processing.
3) Uploading of processed data to the cloud for storage and analysis.
4) Data visualization for seat occupancy, individual seat status and coordinates.
5) Android Application that offers a unified experience with website for observing the current seat availability.

Architecture

1. Image Processing- The technicalities and the dynamic of the image processing algorithms are discussed below. The processing algorithm utilizes Haar features for training accuracy and their robustness. In Haar like feature set, there are multiple features. Few of them are shown below.

The processing steps used result in three different cases of operation: a. When we have multiple seats at the fixed position: This is the most basic case, where the position of the chairs are assumed to be fixed. Then the detection if the seat is empty or not is based on face detection algorithm based on Haar-like cascade features. This isn’t a real-life scenario, as chairs aren’t generally fixed, but movable. We tackled this problem in part-b of this project. b. Single moving chair: In this case, the position of the single chair is determined position, and it is tracked using the color detection algorithm. Then the occupancy is determined based on the same face detection method as discussed above. Then upon movement of the chair, the frame of detection of the chair is changed from fixed to dynamic, which can be further explored for various applications. c. Multiple fixed chairs: This scenario is much closer to real-life scenario. In this case, positions of the chairs are fixed and differentiation between the positions is done, based on color based detection. Also, face detection algorithms are implemented to detect the occupancy of the chairs. This case can be further explored by combining it with case b, where we have multiple moving chairs that can be tracked based on color based tracking algorithm. This can be considered as very close to real-life scenarios.

2. Server- Flask App and MongoDB database: FLASK app is made based on the FLASK framework for python which helps to design a RESTful API to send and receive data through HTTP requests. The FLASK app specifies independent app routes for handling POST requests from the Raspberry Pi which sends in the processed data in JSON format. The data such as the coordinates of the chair, Occupancy status, timestamp (to indicate time instance at which we are taking the information) and seat number of the chairs are parsed and stored in a MongoDB database. The data stored can be visualized through a web application that uses HTTP GET requests for requesting the data stored in MongoDB to a HTML template for rendering dynamic HighCharts.

3. Android Application and Data Analytics: Android Application is one of frontend interface alongside the web app. It also includes the data visualization in the form of dynamic graphs similar to that offered in the web app. The android application also indicates seat occupancy status at given time, percentage occupancy and even the coordinates of the chair.

Technical Components

Raspberry Pi: The Raspberry Pi is a series of credit card-sized single-board computers developed in the United Kingdom by the Raspberry Pi Foundation. The Raspberry Pi 3 is the third-generation Raspberry Pi and it has better connectivity and much improved specs over its predecessors. Its hardware specifications include a 64-bit ARM based processor and 1 GB RAM. It can run a variety of desktop class OS and applications. One of the many supported operating systems is the Raspbian OS, a Linux distribution. It can be used in embedded applications through its versatile interfacing and connectivity . It has 40 GPIO pins, HDMI input, 4 USB inputs, Audio jack and Ethernet port. It supports the latest versions of both Bluetooth and Wi-Fi and it is therefore IoT ready.

USB Camera: The portable USB live webcam by Creative is used for capturing live feed of images. It supports recording at a frame rate of upto 30fps and 720p. it additionally has built in noise canceller for minimizing background noise and interference. It has install clip for mounting on a stable video frame or a pedestal.

AWS EC2: Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers by allowing users to rent virtual computers to run their own computer applications. EC2 encourages scalable deployment of applications by providing a web service through which a user can boot an Amazon Machine Image (AMI) to configure a virtual machine, which Amazon calls an "instance", containing any software desired. A user can create, launch, and terminate server-instances as needed, paying by the hour for active servers – hence the term "elastic". EC2 provides users with control over the geographical location of instances that allows for latency optimization and high levels of redundancy. An Ubuntu 14.04 AMI is used through AWS EC2 for creating and running the FLASK app, storing received data through MongoDb and hosting the web application.

Prototype

In this Seat occupancy detection system, we basically started off with a simple prototype of seat occupancy detection for single seat at a fixed position. Then, after the successful detection of the occupancy of the single seat using face detection algorithm based on Haar-like cascade features, we then went on to work on the similar mechanism for multiple seats with fixed position.
Then the next case was to work on the movement of the seat and detecting its movement and position using the color based detection. In color-based detection, we put different colored sheets on multiple seats and then use that as a differentiating component to determine the position of the seats. Then we adjusted our frame of detection of chair accordingly, to cope up with the movement of the seat. This is basically our second case.The color of the sheets signifies the color of the seat. As this is our first prototype, we took the liberty of attaching, colored sheets in order to develop and test a tracking algorithm.
Then in third case, the first two cases were combined to make a system for more than a single seat and fort determining both the position and occupancy of the seats. From this system in the third part, we figured out that multiple colored tags will be needed to detect multiple seats in the library or auditorium, which will very likely be the biggest problem of our system, typically when it comes to scaling.
Hence, we concluded that since this system is a prototype model of a long term, more accurate and robust system that we want to build, we need to go and implement higher level image processing and machine learning techniques, such as Neural Networks. In Neural Networks, we can try to make the system more accurate, by working more on the classifiers and boosting the necessary and strong ones and taking out the weak ones (more like the refined version of Adaboost algorithm).

http://www.nlpr.ia.ac.cn/2012papers/gnhy/nh14.pdf

https://docs.google.com/viewer?url=patentimages.storage.googleapis.com/pdfs/US20120242492.pdf

http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=4745783

http://www.isprs.org/proceedings/XXXIII/congress/part5/230_XXXIII-part5.pdf

http://events.linuxfoundation.org/sites/events/files/alss13_erencaliskan.pdf

http://www.patentsencyclopedia.com/app/20160092733