Should You Use Android Devices as a Robot's Brains?
Azoft Blog Should You Use Android Devices as a Robot’s Brains?

Should You Use Android Devices as a Robot’s Brains?

By Nikita Krichfalushiy on August 28, 2014

using android smartphones as robot brains

If your robot does nothing but move around, it may not require sophisticated software at all. However, as its functionality becomes complicated, e.g. moving after recognizable objects, the robot CPU may not be able to cope with the task.

We did some experiments in which we used Android devices to control a robot. Our results show that it works, if:

  • Performance is not critical.
  • Task focus is narrow enough.

In this article we’ll describe how to build and program a robot which can recognize a human or any other target (using shape and color) and follow the target. Look down the end of article for performance tests using various Android devices by HTC, Huawei and Samsung.

Disclaimer: We only used a few devices at hand. It may be possible that performance and multitasking problems will be of little or no concern if better hardware is used.

The procedure

Building the robot

To assemble an Android-controlled robot which can recognize targets and follow them, make sure you have:

  • Android device with specially written software
  • Arduino to minimize soldering
  • Bluetooth board
  • Sensors
  • Motors

using android smartphones as robot brains 01
Image 1. Project architecture

components connection diagram
Image 2. Components connection diagram


The software part includes an Android application and Arduino firmware.

Android application

Android application modules

  • MainActivity – choosing robot control mode.
  • ActivityCamera – choosing target identification mode; calculating wheel movement speeds.
  • ColorBlobDetector – helper module to look for outlines by color.
  • А-device can be used as remote control, which is not very interesting.
  • ActivityAccelerometr – polling positioning sensors; calculating wheel movement speeds.
  • BluetoothManager – singleton module to communicate with the robot using phone Bluetooth connectivity.

Searching targets

Using computer vision, the robot can recognize a target and follow it. This functionality is implemented with OpenCV, a rich and powerful library for image recognition which has an Android port.

Possible targets may be: a label in the shape of a circle, object of arbitrary color or human (head and shoulders silhouette).

We use the following algorithms to look for the target in a captured image:

  • Color detection
  • Canny edge detection
  • Haar cascades-based classifier

The algorithm returns an array of all detected outlines which is passed down to outline processing algorithm. The result of the latter algorithm is position of target label in a frame if present. Outline processing algorithm consists of the following steps:

  • Area and perimeter is calculated for every outline.
  • Outlines with area less than 0.25% of frame area are discarded.
  • An outline with maximum area is selected among remaining outlines.

Canny edge detection algorithm is used to look circle-like targets.

Haar cascades-based classifier shipped with OpenCV is used to look for human-like targets.

Android application components

Android application component diagram
Image 3. Android application component diagram

  • Android Environment – hardware components and software framework for mobile application development running under Android OS, including camera, display, Bluetooth connectivity and SDK for interfacing with the OS;
  • JRE – Java Runtime Environment;
  • Android Application – application to control the robot;
  • OpenCV– computer vision library;
  • UIControllers – user interface event handlers;
  • UILayouts – user interface layouts;
  • Localization Strings;
  • Recognition Components;
  • Bluetooth Communication.

Arduino application components

Arduino application component diagram

Image 4. Arduino application component diagram

  • Arduino Application – Arduino sketch to interact with the Android device and control wheel movement;
  • Bluetooth Component;
  • Motor Component.

Android and Arduino component interaction

Android and Arduino software communicate over Bluetooth. Android application sends commands to Arduino in the form of a string:

L-155\rR-155\r, (where numbers can be any number from 0 to 255)

L and R characters are prefixes used to tell Arduino which motor is about to change the speed. Next character is either “-” or “+” which specifies whether reverse or forward rotation should be triggered. Carriage return character is used to separate commands for different motors. Any other data is discarded by the receiver.


We tested the algorithms on several Android devices using a static picture. The performance of the algorithms on various devices is summarized in the following table.

the performance comparison of the algorithms on various devices

benchmark test results of the AnTuTu Benchmark app

VN:F [1.9.22_1171]
Rating: 5.0/5 (8 votes cast)
VN:F [1.9.22_1171]
Rating: +6 (from 6 votes)
Should You Use Android Devices as a Robot's Brains?, 5.0 out of 5 based on 8 ratings

Request a Free Quote

Please enter the result and submit the form

Content created by Nikita Krichfalushiy