Real-time Fuzzy Image Processing Method for Visual feedback of a Follower Robot
一种用于跟随机器人视觉反馈的实时模糊图像处理方法

MOHSEN DAVOUDI, AFSHIN HEIDARIAN
MOHSENDAVOUDI，阿夫辛·海达里安

Department of Electrical Engineering Imam Khomeini International University University Blvd., Qazvin,
电气工程系 伊玛目霍梅尼国际大学大学大道， 加兹温，

Abstract: - In this paper, an adaptive visual feedback system and controller has been designed and implemented in real-time to control the movements of a line follower robot to be smoother and faster. The robot consists of a couple of motorized wheels, the real-time controller and a CMOS camera as the only sensor for detecting line and feedback. The measurement based on real-time image processing and motor drive feedback used in this robot makes it robust to the obstacles and surface disturbances that may deviate robot. The image processing algorithm is adaptive to the line’s color and width too. Image processing techniques have been implemented in real-time to detect the line in the image frame and extract the necessary information (like line’s edge, coordinates and angle). A NI myRIO module is used as a stand-alone hardware unit and RT (Real-Time) target for implementation of controllers and image processing in LabVIEW environment. Both results of real-time and non-real-time implementation of controllers have been compared. To show the performance of real-time image processing in the control of this robot, three types of controllers (i.e. P, PI and Fuzzy controllers) have been implemented for line following tests and the results have been compared. At the end, it was found that the fuzzy controller controls the robot movements smoother, faster, with less errors and quicker response time compare to the other controllers.
摘要： - 本文设计并实时实现了自适应视觉反馈系统和控制器，以控制巡线机器人的运动更平稳、更快速。该机器人由几个电动轮子、实时控制器和一个 CMOS 摄像头组成，作为检测线路和反馈的唯一传感器。该机器人基于实时图像处理和电机驱动反馈的测量使其对可能偏离机器人的障碍物和表面干扰具有很强的抵抗力。图像处理算法也可以适应线条的颜色和宽度。已经实现了实时图像处理技术，以检测图像帧中的线条并提取必要的信息（如线条的边缘、坐标和角度）。NI myRIO 模 块 用作 独立 硬件 单元 和 RT （Real- Time） 目标，用于在 LabVIEW 环境中 实现 控制器 和 图像 处理。 对控制器的实时和非实时实现的结果进行了比较。为了展示实时图像处理在该机器人控制中的性能，我们实施了三种类型的控制器（即 P、PI 和 Fuzzy 控制器）进行巡线测试，并对结果进行了比较。最后，发现模糊控制器控制机器人的运动比其他控制器更平稳、更快、误差更小、响应时间更快控制器。

Key-Words: - Real-time, Image Processing, Fuzzy Logic, Visual Feedback, Angle measurement, Follower Robot
关键词：-实时，图像处理，模糊逻辑，视觉反馈，角度测量，跟随机器人

Received: October 18, 2019. Revised: February 13, 2020. Accepted: February 29, 2020. Published: March 15, 2020.
收稿日期： 2019-10-18.修订日期：2020 年 2 月 13 日。录用日期：2020 年 2 月 29 日。发布时间：2020 年 3 月 15 日。

Introduction
介绍

In the past decades, as the robotics became more popular, the robots have been divided into the following categories in terms of application and movements: Manipulators, wheeled robots, flying robots and human-like robot [1]. Line follower robots are robots capable of tracing a colored line possibly different to the background color. Most of the line followers are using a number of light-sensitive sensors to detect the line [2]. Most of the time the accuracy of measurement by using light-sensitive sensors is low because the accuracy is affected by poor reflectivity of the ground plane and the noisy light and some nonlinearities. On the other hand, the high number of light-sensitive sensors with special arrangement causes the robot to be bigger in size and less ability to maneuver. Machine vision has been developed to solve this problem, but the need for large process capacity for real-time image processing has been the bottleneck of this approach. Improving the real-time image processing methods for visual feedback and visual servoing in robotics are now an essential in control of the robots The software for simulation and design of real-time visual feedback
在过去的几十年里，随着机器人技术的日益普及，机器人在应用和运动方面分为以下几类：机械手、轮式机器人、飞行机器人和类人机器人[1]。线条跟随机器人是能够追踪可能与背景颜色不同的彩色线条的机器人。大多数线路跟随者正在使用许多光敏传感器来检测线路 [2]。大多数情况下，使用光敏传感器的测量精度较低，因为精度会受到接地平面反射率差、嘈杂光和一些非线性的影响。另一方面，大量具有特殊布置的光敏传感器导致机器人体积更大，机动能力更差。机器视觉已经开发出来解决这个问题，但实时图像处理对大处理能力的需求一直是这种方法的瓶颈。改进机器人技术中视觉反馈和视觉伺服的实时图像处理方法现在是控制机器人的必要条件。用于实时视觉反馈的仿真和设计软件

and motor control algorithms is LabVIEW that provides a graphical programming environment which is broadly used for different applications in industry, education, training and laboratory research [3].
电机控制算法是 LabVIEW，它提供了一个图形编程环境，广泛用于工业、教育、培训和实验室研究中的不同应用 [3]。

Using image processing instead of light sensitive sensors to detect the line gives the robot this capability to choose and follow the lines with targeted colors and gives this possibly to change the robot’s path by changing the desired line’s color or by changing the target color in software. Also it would be possible to have a line with different colors in order to control the robot’s speed using a simple controller. This would be useful in the applications that the line’s color indicates the speed limit or any caution.
使用图像处理而不是光敏传感器来检测线条，使机器人能够选择和跟随具有目标颜色的线条，并且可以通过更改所需线条的颜色或在软件中更改目标颜色来改变机器人的路径。此外，还可以有一条不同颜色的线，以便使用简单的控制器来控制机器人的速度。这在线条的颜色指示速度限制或任何警告的应用程序中非常有用。

The real-time control algorithm runs on a NI myRIO board which has the capability to run the programs in real-time. The aim of writing the program in real time is to manage the tasks and hardware by giving the priority to each tasks in order to reduce the robot fails while following the targeted line by maximizing the frames to be taken and processed [4]. The experimental results show that among three types of
实时控制算法在NImyRIO 板 卡 上运行， 该 板 卡 能够 实时 运行程序。实时编写程序的目的是通过优先考虑每个任务来管理任务和硬件，以便通过最大化要获取和处理的帧来减少机器人在遵循目标路线的同时失败 [4]。实验结果表明，在3 种类型的

controller which all have been implemented in real- time, the fuzzy logic-based controller has smoother movements while the overshoot and oscillation of the robot angular degree has been minimized.
控制器，基于模糊逻辑的控制器具有更流畅的运动，同时最大限度地减少了机器人角度的过冲和振荡。

Fig.1: Top)The different paths which have been distinguished by their color. Five colors green, blue, yellow, red and black have been used in this sketch.
图 1：上）按颜色区分的不同路径。这幅素描中使用了五种颜色：绿色、蓝色、黄色、红色和黑色。

Bottom) Robot’s plane for testing.
下）用于测试的机器人飞机。

Real-time Image Measurements
实时图像测量

To detect the targeted colored line to be followed by the controller of the robot, the image processing techniques has been used. The first step is to get and store the image (one frame of the video) from the robot’s CMOS camera with the resolution of 640*480. Higher resolution makes the angle measurement more accurate but it makes the time longer for image processing and as a result the number of frames per second reduces .The camera is connected via USB port to myRIO board. There are some functions for image processing including color detection, function, various mathematical filters for, noise reduction, angle detection, diagnosis, etc. Although there are some ready functions for abovementioned tasks, due to their long delay and uncertainty have not been used in real-time tasks. So
为了检测机器人控制器要跟踪的目标彩色线，使用了图像处理技术。第一步是从机器人的 CMOS 相机获取并存储图像（视频的一帧），分辨率为 640*480。更高的分辨率使角度测量更准确，但它会使图像处理的时间更长，因此每秒的帧数会减少。相机通过 USB 端口连接到 myRIO 板。图像处理有一些功能，包括颜色检测、功能、各种数学滤波器、降噪、角度检测、诊断等。虽然上述任务有一些现成的函数，但由于它们的延迟时间长和不确定性，尚未在实时任务中使用。所以

those tasks have been all programmed using basic functions.
这些任务都是使用基本功能编程的。

Image Processing
图像处理

As previously mentioned, the line to be detected by the robot has a distinct color compared to its background. The difference of RGB color levels is used for detecting the line. The function used for detection of the colored line in LabVIEW image processing is “IMAQ Color Threshold” function [5]. In this function by determining the amount of red, blue and green color which are numbers between 0 and 255, the targeted color of the line can be determined.
如前所述，与背景相比，机器人要检测的线条具有独特的颜色。RGB 色阶的差异用于检测线条。在 LabVIEW 图像处理中用于检测彩色线条的函数是“IMAQ颜色阈值”函数 [5]。在这个函数中，通过确定红色、蓝色和绿色（介于 0 和 255 之间的数字）的数量，可以确定线条的目标颜色。

In the next step, the extracted image consisting of the targeted color is changed from RGB mode to binary or two-colored mode. Meaning that the areas that consist of our desired color must be specified by a distinguished color that we can choose, like red, and the other areas are displayed in black, which means that these areas don’t have our desired color and they have been removed from the image.
在下一步中，由目标颜色组成的提取图像从 RGB 模式更改为二进制或双色模式。这意味着由我们所需的颜色组成的区域必须由我们可以选择的可区分颜色（如红色）指定，而其他区域显示为黑色，这意味着这些区域没有我们想要的颜色，并且它们已从图像中删除。

Fig. 2: This Fig. shows a black line located in a dirty plane, by using image processing all the noises is removed and it changed to the red line and the necessary information like line’s edges and width coordinates is extracted from it as outputs.
图 2：该图显示了一条位于脏平面上的黑线，通过使用图像处理，所有噪声都被去除并变为红线，并从中提取必要的信息，如线的边缘和宽度坐标作为输出。

Using some filter functions improve the two-colored image cleanness causing the extracted data be more accurate. The next function is the IMAQ Particle Filter function which can remove the values identified in the previous step based on the size, shape, length, width and other characteristics. Using this function, removes the potential points and spots on the ground floor, which have the same color as the line (see Fig. 2). The next step consists of extracting
使用一些过滤器函数可以提高双色图像的清晰度，从而使提取的数据更加准确。下一个功能是 IMAQ Particle Filter 功能，它可以根据大小、形状、长度、宽度和其他特征删除上一步中确定的值。使用此功能，去除一楼上与线条颜色相同的潜在点和点（见图 D）。2）.下一步包括提取

the desirable information that is needed for controlling the robot. The IMAQ Particle Analysis function has important capabilities such as detecting the angle, the center of the shape, shape’s area, etc. This step is one of the main steps of the image processing algorithm in which only the necessary information should be extracted from the image; otherwise, it increases the quantity of calculation and cause’s some delays in running the algorithm [5].
控制机器人所需的所需信息。IMAQ Particle Analysis功能具有检测角度、形状中心、形状面积等重要功能。此步骤是图像处理算法的主要步骤之一，其中只应从图像中提取必要的信息;否则，会增加计算量并导致算法运行的一些延迟 [5]。

Angle Measurement Algorithm
角度测量算法

After detecting the line and its edges it is time to use these data to measure the line’s angle. Fig. 3 shows an image frame of the camera (the pink area) which the part of the line that located in the frame is determined by different color. This part of the targeted line at least has four edges because the camera frame has cut the line while it could have more edges in facing with broken targeted lines too. In order to choose the right edges, it is necessary to define some rules. In fact, the four blue spots seen in the pink area of the Fig. 3, i.e. (𝑥₁, 𝑦₁) to (𝑥₄, 𝑦₄), should be chosen correctly in any condition of the targeted line like straight line, turn left and turn right.
检测出线条及其边缘后，就可以使用这些数据来测量线条的角度了。图 3 显示了相机的图像帧（粉红色区域），位于帧中的线部分由不同的颜色决定。目标线的这一部分至少有四个边缘，因为相机帧已经切断了线，而面对虚线时也可能有更多的边缘。为了选择正确的边缘，有必要定义一些规则。事实上，图 3 粉红色区域中看到的四个蓝色点，即 （x₁y₁） 到 （x₄y₄），在目标线的任何条件下都应该正确选择，例如直线、左转和右转。

The next step in measuring the targeted line’s angle is to calculate the (𝑋₁. 𝑌₁) and (𝑋₂. 𝑌₂) (the red points in Fig. 3). Here are the equations for calculation of those two points.
测量目标线角度的下一步是计算 （X₁Y₁） 和 （X₂Y₂）（图 3 中的红点）。以下是计算这两点的方程式。

𝑋₁ = 𝑥₁ − 𝑥₂
X₁=x₁− _{x 2}

𝑌₁ = 𝑦₁ = 𝑦₂ (1)
Y₁=Y₁=Y₂（1）

𝑋₂ = 𝑥₃ = 𝑥₄
X₂=x₃ =x₄

𝑌₂ = 𝑦₃ − 𝑦₄ (2)
Y₂=Y₃−Y₄（2）

The absolute value of 𝑋₁ has been used for the next steps. These two points are used in the final step to measure the targeted line angle. A line between (𝑋₁. 𝑌₁) and (𝑋₂. 𝑌₂) is drawn to create the θ angle with the horizontal line, which is considered as the angle of the targeted line seen in the frame of the camera to be measured using equation 3.
X₁ 的绝对值已用于后续步骤。这两个点在最后一步用于测量目标线角度。在 （X₁Y₁） 和 （X₂Y₂） 之间绘制一条线，以使用水平线创建 θ 角，该水平线被视为在相机框架中看到的目标线的角度，使用公式 3 进行测量。

𝑋₂ − 𝑋₁
X 2-X ₁

(3)
（3）

Fig. 3: A view of the part of the line seen in the frame and the edges needed for angle measurement.
无花果。3：在框架中看到的线部分和角度测量所需的边缘的视图。

There is the possibility that the (𝑥₁. 𝑦₁), (𝑥₂. 𝑦₂) spots could be chosen instead of each other but in the angle measurement process it will be clear that it does not make any problem.
有可能选择（x₁y₁）， （x₂y₂）点而不是彼此，但在角度测量过程中很明显它不会造成任何问题。

Table 1: The spots which is needed for calculating the angle and the rules for choosing them among the other edges
表1：计算角度所需的光斑以及在其他边缘中选择光斑的规则

This algorithm calculates the angle from 0 to 180 degree, but it is not symmetric and also it calculates the straight line’s angle 90 degree which it is better to be zero to make the line follower robot to be controlled easier so 90 degree is decrease from the angle that calculated from the equation 3.
该算法计算 0 到 180 度的角度，但它不是对称的，而且它计算直线的角度为 90度，最好为零以使巡线机器人更容易控制，因此 90 度从从公式 3 计算的角度减小。

Line Follower Design
巡线器设计

The sample robot, chosen to implement the proposed algorithm, is a two-wheeled robot. The Motors used in this robot are couple of DC motors with a maximum speed of 20 rpm, whose velocities are controlled using PWM (pulse width modulation) amplitude signals generated directly by myRIO. For supplying the myRIO board and bridge drivers a DC power supply of 12 volts or a 12-volt battery can be used.
选择实现所提算法的示例机器人是一个两轮机器人。该机器人中使用的电机是一对直流电机，最大速度为 20 rpm，其速度使用 myRIO 直接生成的 PWM（脉宽调制）幅度信号进行控制。为了给myRIO板 卡和桥驱动器 供电， 可 使用 12 伏 直流 电源 或 12 伏 电池。

MyRIO board has the capability to setup a webcam (connected via USB port) [6] and also has the capability to connect to a computer via Wi-Fi. The PWM outputs of the myRIO board is used for controlling the robot motors.
MyRIO 板 板 能够 设置 网络摄像头 （通过 USB 端口 连接） [6] ， 还 能够 通过 Wi-Fi 连接 到 计算机。myRIO 板的 PWM 输出 用于 控制 机器人 电机。

Using the myRIO board makes it possible to monitor the robot wirelessly at any time, for example displaying the PWM signal of each motor, the image
使用myRIO板卡，可以随时无线监控机器人，例如显示每个电机的PWM信号、图像

of the detected line, the measured angle, the frame of the image, etc. on the desktop version of the written program on PC [7], [8]. There is also the possibility to modify the controller method, run and stop the robot and choose the targeted line color that the robot should follow at any moment by the computer via the wireless connection while the robot is running and also there is a capability to improve the program like controller parameters or image processing algorithm wirelessly while the program is not running. This feature makes the process of changing and testing the program easier.
在桌面版PC上编写的程序[7]、[8]上检测到的线、测得的角度、图像的帧数等。还可以修改控制器方法，运行和停止机器人，并在机器人运行时通过无线连接选择机器人应随时由计算机遵循的目标线条颜色，并且还具有功能在程序不运行时无线改进程序，如控制器参数或图像处理算法。此功能使更改和测试程序的过程更加容易。

Since the controller can’t supply the power needed by the motors, a PWM signal amplifier has been designed using a L298N bridge driver to switch the required power to the motors.
由于控制器无法提供电机所需的功率，因此设计了一个 PWM 信号放大器，使用 L298N 桥式驱动器将所需的功率切换到电机。

Real-Time Controller
实时控制器

One of the important features of the controllers and visual feedback algorithms, which is running on this robot, is the real-time principles applied to its tasks. This means that the tasks are prioritized and run according to their priority in a fixed time period. In fact, tasks are run before passing the deadline so that no disruptions or fails would occur while the robot is being controlled using the proposed algorithm. The idea of designing the controller to run in real-time is due to the fact that a certain number of image frames to be processed to extract the information needed for the fuzzy controller which makes the control more reliable and ensures the movements of the robot to be smooth especially in the line turns.
在该机器人上运行的控制器和视觉反馈算法的重要功能之一是应用于其任务的实时原理。这意味着任务在固定时间段内根据其优先级确定优先级并运行。事实上，任务是在截止日期之前运行的，因此在使用建议的算法控制机器人时不会发生中断或失败。将控制器设计为实时运行的想法是由于要处理一定数量的图像帧以提取模糊控制器所需的信息，这使得控制更加可靠并确保机器人的运动要平稳，尤其是在线路转弯时。

To implement the control algorithm in real-time, the different parts of the graphical code in LabVIEW is considered as a task that executes in a certain time period, then by creating a subVI for each of these tasks, different parts of the control algorithm such as commending the robot, image processing part and control algorithm (such as Fuzzy, P, PI) are all separated and prioritized depending on the importance of its role they play in controlling the robot and the time needed to execution. Table 2 shows all of the subVIs and their dedicated priorities. In this table the smaller the number the higher the priority [9].
为了实时实现控制算法，将LabVIEW中图形代码的不同部分视为在一定时间内执行的任务，然后通过为每个任务创建一个子VI，控制算法的不同部分，例如推荐机器人，图像处理部分和控制算法（例如Fuzzy， P、PI）都被分离并确定其优先级，具体取决于它们在控制机器人中的作用的重要性以及执行所需的时间。表 2 显示了所有子VI及其专用优先级。在此表中，数字越小，优先级越高 [9]。

Table 2: The SubVIs of the real-time control algorithm and the priorities assigned to each of them
表 2：实时控制算法的子 VI 以及分配给每个算法的优先级

Another important issue to be considered while coding the visual feedback and controller in real-time is the data flow among the prioritized tasks. For correct implementing the data flow among the prioritized subVIs instead of using “wire” in LabVIEW, specific functions for sending and receiving the data should be used called “global variables” and “Real-time FIFO” functions. In myRIO, for making a set of sub VIs to run in real- time four hardware/software components are needed:
在实时编码视觉反馈和控制器时要考虑的另一个重要问题是优先任务之间的数据流。为了正确实现优先子VI之间的数据流，而不是在LabVIEW中使用“wire”，应使用用于发送和接收数据的特定函数，称为“全局变量”和“Real-time FIFO”函数。在 myRIO 中，要制作一组实时运行的子 VI，需要四个硬件/软件组件：

1) Host computer that runs the monitoring and front panel programs, 2) Real-time module that is a software module to arrange the sub VIs to run in real- time, 3) RT target that runs the visual feedback and controller algorithms in real-time and 4) RT engine that is a real-time LabVIEW version [9].
1） 运行监控和前面板程序的主机，2） 实时模块，用于安排子VI 实时运行的软件模块，3） 实时运行视觉反馈和控制器算法的 RT 目标，以及 4） 实时 LabVIEW 版本的 RT 引擎 [9]。

To describe more in detail the structure of abovementioned hardware and software set, it can be said that the host computer is a computer that designs
为了更详细地描述上述硬件和软件集的结构，可以说主机是一台设计

VI which can have operating systems such as Windows, Linux, and IOS. RT targets are a collection of hardware devices which have the capability to run an RT engine. In fact, these hardware devices are suitable for running software programs in real-time while they don’t have the hardware limits and delays seen in normal computers with operating system running a program. RT targets are divided into plug- in devices and RT network series [9].
VI 可以具有 Windows、Linux 和IOS 等操作系统。RT目标是一组能够运行 RT 引擎的硬件设备。事实上，这些硬件设备适合实时运行软件程序，而它们没有操作系统运行程序的普通计算机中所见的硬件限制和延迟。RT 目标分为插件设备和 RT 网络系列 [9]。

Fig. 4: A view of communication of RT target and Host computer and the part of the LabVIEW they run
图 4： RT 目标和主机的通信视图以及它们运行的 LabVIEW 部分

In the designed robot, the RT target is myRIO board, which is a plug-in device. The RT engine is a LabVIEW version which has been installed on the RT target and basically, its job is to run VIs which are designed by a user through a computer. Since it runs on the RT target, it is able to run the program in real-time [9].
在设计的机器人中，RT目标是myRIO板，它是一个插件设备。RT 引擎是已安装在 RT 终端上的 LabVIEW 版本，基本上，它的工作是通过计算机运行用户设计的 VI。由于它在RT 目标上运行，因此它能够实时运行程序[9]。

Fig. 5: An overall view of the real-time algorithm
无花果。5：实时算法的整体视图

Also, experimental results show that when the program, run in a non-real-time framework, the number of video frames can be processed are among
此外，实验结果表明，当程序在非实时框架下运行时，可以处理的视频帧数在

18 to 23 frames per second. But the real-time framework has the advantage of increasing the number of video frames to be processed higher (23- 26) frames per second.
每秒 18 到 23 帧。但是实时框架的优势在于将要处理的视频帧数增加到更高的每秒 （23-26） 帧。

In the simulation and experimental part of the work, three types of controlling methods include proportional (P), proportional integration (PI) and Fuzzy logic controller (FLC) have been implemented and as the final experimental results, these three methods have been compared with each other too.
在工作的仿真和实验部分，实现了比例 （P）、比例积分 （PI） 和模糊逻辑控制器 （FLC） 三种类型的控制方法，作为最终的实验结果，这三种方法也进行了相互比较。

P and PI controller
P和PI控制器

Since the design of these two controllers is nearly
由于这两个控制器的设计几乎是

moment and the error (i.e. the difference between set point and the angle of the robot direction) is multiplied to the proportion coefficient, 𝐾𝑝, and is applied to the system [10]. The Fig. 12 shows the block diagram of the PI controller designed for the robot in which R(s) is the input or set point angle, C(s) is the output angle, 𝐾𝑝 is the proportional coefficient, 𝐾_𝐼 is the integral coefficient, H(s) is the feedback ratio and G(s) is the transfer function of the robot system. The priorities assigned to each part of the real-time control loop have been included in this Fig. too. Table 5 shows the coefficients obtained for the PI controller.
力矩和误差（即设定点与机器人方向角度之间的差值）乘以比例系数 Kp，并应用于系统 [10]。图 12 显示了为机器人设计的 PI 控制器的框图，其中 R（s） 是输入或设定点角度，C（s） 是输出角度，Kp 是比例系数，K_I是积分系数，H（s） 是反馈比率，G（s） 是传递函数机器人系统的。分配给实时控制回路每个部分的优先级也包含在此图中。表5 显示了为PI 控制器获得的系数。

Fig. 6: The block diagram of the PI controller designed for the robot including visual feedback and the priorities assigned to each part of the real-time control loop
无花果。6：为机器人设计的PI控制器框图，包括视觉反馈和分配给实时控制回路每个部分的优先级

same, the designing of the PI controller is just explained but the results of both Proportional and PI controllers have been compared by each other at the
相同，仅解释了 PI 控制器的设计，但比例控制器和PI 控制器的结果已在

Table 4: KP
表4：KP

and K
和 KI

coefficients of the PI controller
PI控制器的系数

end to show the effects of integral coefficient, 𝐾_𝐼
end显示积分系数K_I 的效应.

In order to design PI controller, the range of the angles that the robot makes with the targeted line has been extracted from some experiments. The characteristics of the robot used in this study is generally not linear, but in some determined ranges, it can be piecewise linearized. Since the control method is not suitable for nonlinear systems, a controller has been designed for each range of angles that the robot shows nearly linear behavior. Table 3 shows the different ranges.
为了设计 PI 控制器，从一些实验中提取了机器人与目标线的角度范围。本研究中使用的机器人的特性通常不是线性的，但在一些确定的范围内，它可以分段线性化。由于该控制方法不适用于非线性系统，因此为机器人表现出近线性行为的每个角度范围设计了一个控制器。表 3 显示了不同的范围。

Table 3: The ranges that the Proportional and PI controllers shows nearly linear behavior.
表3：Proportional和 PI 控制器显示近乎线性行为的范围。

The implemented controller is a PI controller, a feedback from the angle is applied to the control
实现的控制器是一个PI控制器，来自该角度的反馈被施加到控制器上

The desired set point angle is 0 degree which is applied to the input of the controller (i.e. R(s)). Thus, the output angle is reduced from zero at every
所需的设定点角度为 0 度，应用于控制器的输入（即 R（s））。因此，输出角度在每个

Table 5: KP
表5：KP

Coefficients of the line width coordinate proportional controllers
线宽坐标比例控制器的系数

For simplicity, this control can be done when the robot follows a straight path. The measuring range of the line width coordinate is in the range of 0cm to
为简单起见，当机器人沿着直线路径时，可以进行这种控制。线宽坐标的测量范围在0cm到

19.5cm and the desired range is 1.8cm to 16.5cm. To control the line’s width coordinate. The variable is specified in two ranges: 1) Straight lines toward the left, and 2) Straight lines toward the right. A couple of Proportional closed-loop controllers are designed for both ranges.
19.5 厘米，所需范围为1.8 厘米至16.5 厘米。控制线条的宽度坐标。该变量在两个范围内指定：1） 向左的直线，以及 2） 向右的直线。两个范围都设计了几个比例闭环控制器。

Fig. 7: A view of the line width coordinates controlling method
图 7： 线宽坐标控制方法的视图

Fuzzy controller
模糊控制器

To implement the fuzzy logic-based controller for the line follower robot a Fuzzy System Designer function has been used in the LabVIEW environment which is similar to the Fuzzy logic tool in MATLAB[11].
为了实现基于模糊逻辑的控制器，为巡线机器人，在LabVIEW环境中使用了FuzzySystemDesigner函数，该函数类似于 MATLAB 中的模糊逻辑工具[11]。

The first step in designing the Fuzzy controller is to introduce the input and output membership functions. For simplification, by specifying different states of the robot angle direction to the targeted line, three triangular-trapezoidal input membership functions have been introduced for the line angle: 1) Left, 2) Straight and 3) Right. Fig. 8 (a). Shows the line angle input membership functions.
设计 Fuzzy 控制器的第一步是引入input和outputmembership函数。为简化起见，通过指定机器人角度方向与目标线的不同状态，为线角度引入了三个三角形-梯形输入隶属函数：1） 左，2） 直线和3） 右。无花果。8（a） 的。显示线角度输入隶属度函数。

Fig. 8: Fuzzy membership functions and 3D view of the fuzzy controller results.
图 8： 模糊隶属函数和模糊控制器结果的 3D 视图。

As noted in the previous section, in addition to controlling the angle of the robot, line width coordinates is also necessary to be measured and used
如上一节所述，除了控制机器人的角度外，还需要测量和使用线宽坐标

in controlling the robot. Thus, line width coordinate membership functions of the line must be introduced for different states. Fig. 8(b). Shows the line width coordination input membership functions. Red membership function represents the state that the line is located in the left side of axle between the wheels, blue membership function represents the state that the line is in the desired region (center) between the wheels and the green membership function is related to the state in which the line is located in the right side of axle of wheels.
在控制机器人。因此，必须为不同的状态引入线的线宽坐标隶属函数。图 8（b）。显示线宽协调输入隶属度函数。红色隶属函数表示线条位于车轮之间的车轴左侧的状态，蓝色隶属函数表示线条位于车轮之间所需区域（中心）的状态，绿色隶属函数为与线位于车轮轴右侧的状态有关。

The outputs are normalized in order to get the right PWM signal for right and left motors using the standard functions to be able to use the full range of the DC motors operation.
输出经过标准化，以便使用标准功能为左右电机获得正确的 PWM 信号，以便能够使用直流电机的全部运行范围。

Fig. 8(c). Shows the output membership function of the left motor, the green and yellow membership functions represent the cases that the line width coordinates are in the left and right side of the wheels’ axis respectively, and blue and red colors are for the states which the line leans to the left and right respectively. The output membership functions of the right motor is nearly same as the left one.
图 8（c）。显示左侧电机的输出隶属函数，绿色和黄色隶属函数分别表示线宽坐标在轮子轴的左侧和右侧的情况，蓝色和红色表示线向左倾斜的状态，对。右侧电机的输出隶属函数与左侧电机几乎相同。

One of the objects of designing the different controllers (i.e. P, PI and Fuzzy) in this project is to make the robot to work at its maximum speed and minimum oscillation/vibration. One of the reasons that a gap exists in the Fuzzy output membership functions between 0.2 to 0.9 is this. In fact, it is desired to increase the PWM averaged amplitude signal as much as possible in order to improve the robot’s operation. Another reason of the gap existence is that the speed changes of the motors is not linearly proportional to the changes of the PWM averaged amplitude signal, while it is more sensitive in lower amplitudes rather than the medium amplitudes.
在这个项目中设计不同的控制器（即 P、PI 和 Fuzzy）的目标之一是使机器人以其最大速度和最小的振荡/振动工作。模糊输出隶属函数中存在 0.2 到 0.9 之间的差距的原因之一是。事实上，为了改善机器人的操作，希望尽可能增加 PWM 平均振幅信号。间隙存在的另一个原因是电机的速度变化与 PWM 平均振幅信号的变化不成线性比例，而它在较低振幅而不是中等振幅中更敏感。

After introducing the input and output membership functions, it is time to determine Fuzzy Rules. This step actually relates the input and output states. Table 6 shows the fuzzy rules of the left motor. The fuzzy rules cover all possible conditions of the inputs.
在介绍了输入和输出成员资格函数之后，是时候确定 Fuzzy Rules 了。此步骤实际上将输入和输出状态关联起来。表 6 显示了左侧电机的模糊规则。模糊规则涵盖 Inputs 的所有可能条件。

Table 6: Fuzzy rule-base for the left motor
表6：左侧电机的模糊规则库

Experimental Results n To compare the experimental results of the three control methods implemented (i.e. P, PI and Fuzzy),
实验结果 n为了比较所实施的三种控制方法（即 P、PI 和 Fuzzy）的实验结果，

the performance of the robot controllers for two angles of 90 degrees and 50 degrees have been analyzed, and the data samples have been acquired directly from myRIO to the host computer. The acquired data are saved in .bin files and later opened in MATLAB to draw the plots. It is noted that in the angle control of the robot using PI controller there is a big undershoot and oscillation when the 50-degree turn is performed (see Fig. 9). As shown in Fig. 9, the undershoot became bigger when the turn got sharper (90 degree turn).
已经分析了机器人控制器在 90 度和 50 度两个角度下的性能，并将数据样本直接从 myRIO 采集到主机。采集的数据保存在 .bin文件中，然后在MATLAB 中打开以绘制绘图。值得注意的是，在使用PI控制器的机器人角度控制中，当执行 50 度转弯时，存在较大的下冲和振荡（见图 D）。9）.如图 1 所示。9，当转弯变得更尖锐（90 度转弯）时，下冲变得更大。

Instead, by using the fuzzy controller, the results recorded for both 50 degree turn and 90 degree turn (shown in Fig. 9 and Fig. 10 respectively) show that the undershoot and oscillations are totally cleared. This means that in the turns the robot mimics smooth movements which bring some advantages that comes as follows: 1) The need for smooth movements in the industrial factories that the robot carry liquids are satisfied using fuzzy controller. 2) The mechanical vibrations and shocks reduced very well causing long life operation of the robot. 3) The energy consumption of the robot reduced.
相反，通过使用模糊控制器，记录的 50 度转弯和 90 度转弯的结果（分别如图 9 和图 10 所示）表明下冲和振荡已完全清除。这意味着在转弯时，机器人会模仿平稳的运动，从而带来一些优势，如下所示：1） 在工业工厂中，机器人携带液体的平稳运动需求使用模糊得到满足控制器。2）机械振动和冲击很好地减少，使机器人的使用寿命长。3） 机器人的能耗降低。

50 dgree angle corecction
50度角定心

90 dgree angle corecction
90度角定心

80

70

60

40

30

20

10

0

-10

15 0 5 10 15
票价：1505 1015 元

Time[s]
时间

Fig. 10: 90 degree turn of the robot. Green line is for Angle correction using Proportional controller, Blue line is for Angle correction using PI controller and Red line is for Angle correction using Fuzzy controller .
无花果。机器人 10：90度转弯。绿线用于使用比例控制器进行角度校正，蓝线用于使用 PI 控制器进行角度校正，红线用于使用模糊控制器进行角度校正。

To show the advantage of the fuzzy controller, in another test, the time that the robot needs to finish a constant loop is measured for Proportional, PI and
为了展示模糊控制器的优势，在另一项测试中，测量了机器人完成一个恒定循环所需的时间，以Proportional、PI和

50

40

20

10

0

-10

0 5 10 15

Time[s]
时间

90

80

70

60

40

30

20

10

0

-10

0

Fuzzy controllers (see Fig. 1 (b). The experimental results show fuzzy controller makes the robot quicker than PI controller around 5% and quicker than P controller around 6.5%.
模糊控制器（见图 1 （b）。实验结果表明，模糊控制器使机器人的速度比 PI 控制器快约 5%，比 P 控制器快约 6.5%。

Also, experimental results show that when the program, run in a non-real-time framework, the number of video frames can be processed are among
此外，实验结果表明，当程序在非实时框架下运行时，可以处理的视频帧数在

18 to 23 frames per second. But the real-time framework has the advantage of increasing the number of video frames to be processed higher (23- 26) frames per second.
每秒 18 到 23 帧。但是实时框架的优势在于将要处理的视频帧数增加到更高的每秒 （23-26） 帧。

Conclusions
结论

In this paper, an image processing algorithm for targeted line angle detection and robot alignment has been designed and implemented in real-time to control the movements of a robot. Using visual feedback and fuzzy controller, the line follower robot
本文设计并实现了一种用于目标线角检测和机器人对准的图像处理算法，以实时控制机器人的运动。采用视觉反馈和模糊控制器，线路跟随机器人

Fig. 9: 50 degree turn of the robot. Green line is for Angle
无花果。机器人 9：50度转弯。绿线代表角度

correction using Proportional controller, Blue line is for Angle correction using PI controller and Red line is for Angle correction using Fuzzy controller.
使用比例控制器进行校正，蓝线用于使用 PI 控制器进行角度校正，红线用于使用模糊控制器进行角度校正。

designed in this experiment is being able to follow the line faster while showing smoother movements in turns. The camera is the only feedback device while there is no need to use other position and speed feedback from the wheels in which big errors and uncertainties are incorporated.
本实验的设计是能够更快地跟随路线，同时在轮流中表现出更流畅的动作。摄像头是唯一的反馈设备，而不需要使用来自车轮的其他位置和速度反馈，其中包含了很大的误差和不确定性。

The visual feedback in this robot has an advantage that even if the robot was deviated due to clash of obstacles, it will correct its angle. In this paper the
这款机器人的视觉反馈有一个优势，即使机器人因障碍物碰撞而偏离，它也会纠正其角度。在本文中，

image processing algorithm using P, PI and Fuzzy controllers have been implemented by LabVIEW programming to make the controllers run fully in real-time on a myRIO module. Using the real-time framework to implement the controllers results the control loop run faster. The experimental results show that the fuzzy controller with smart rule defining for such robots has some advantages including faster and smoother movements, less mechanical vibration and shocks, more reliability to carry the liquids, less energy consumption, etc.
LabVIEW 编程实现了使用 P、PI 和 Fuzzy 控制器的图像处理算法，使控制器在 myRIO 模块上完全实时运行。使用实时框架实现控制器可以使控制回路运行得更快。实验结果表明，为此类机器人定义智能规则的模糊控制器具有运动更快、更平稳、机械振动和冲击更小、携带液体更可靠、能耗更低等优点。

References
引用

C.-P. Tsai, C.-T. Chuang, M.-C. Lu, W.-Y.
C.-P.蔡 C.-T.庄 M.-C.卢伟Y.

Wang, S.-F. Su, and S.-L. Chang, “Machine-vision based obstacle avoidance system for robot system,” in System Science and Engineering (ICSSE), 2013
王 S.-F.Su 和 S.-L.Chang，“基于机器视觉的机器人系统避障系统”，系统科学与工程 （ICSSE），2013

Authors
作者

Qazvin, Iran.
伊朗，加兹温。

Mohsen Davoudi received his Ph.D. in Electrical Engineering from Polytechnic University of Milan (Politecnico di Milano), Milan, Italy, in 2011. Currently he has assistant professor position at Imam Khomeini International University (IKIU),
MohsenDavoudi 于 2011 年在意大利米兰理工大学 （Politecnico di Milano） 获得电气工程博士学位。目前，他在伊玛目霍梅尼国际大学 （IKIU） 担任助理教授职位。

Afshin Heidarian received his BSc degree in Electrical Engineering from Shiraz University in 2015. He is presently pursuing Masters in Electrical Engineering at Imam Khomeini International University,
Afshin Heidarian 于2015 年获得设拉子大学电气工程学士学位。他目前正在伊玛目霍梅尼国际大学攻读电气工程硕士学位。

International Conference on, 2013, pp. 273–277.
国际会议，2013 年，第 273-277 页。

N. Chakravarthy and J. Xiao, “FPGA-based control system for miniature robots,” in Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, 2006, pp. 3399–3404.
N. Chakravarthy 和 J. Xiao，“基于 FPGA 的微型机器人控制系统”，《智能机器人与系统》，2006 年 IEEE/RSJ 国际会议，2006 年，第 3399-3404 页。

N. I. L. U. Manual, National Instruments (2003). 2015.
N. I. L. U. 手册，National Instruments （2003）。2015 年。

L. Cuzco, J. Arias, A. Abril, and D. Cabrera, “Acquisition, filtering and decomposition into real- time ECG signal with Ni — MyRio,” in 2015 CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON), 2015, pp. 209–213.
L.Cuzco、J.Arias、A.Abril 和 D.Cabrera，“使用 Ni — MyRio 采集、过滤和分解为实时心电图信号”，2015 年智利电气、电子工程、信息和通信技术会议 （CHILECON），2015 年，第 209-213 页。

R. Posada-gómez, O. O. Sandoval-González,
R.Posada-gómez，O.O.Sandoval-González，

A. Martínez Sibaja, O. Portillo-Rodríguez, and G. Alor-Hernández, “Digital Image Processing Using LabView,” Intechopen, 2001.
A. Martínez Sibaja、O. Portillo-Rodríguez 和 G. Alor-Hernández，“使用 LabView 进行数字图像处理”，Intechopen，2001 年。

National Instruments, “User guide and specification: NI myRIO-1900,” p. 20, 2016.
National Instruments，“用户指南和规范：NI myRIO-1900”，第 20 页，2016 年。

E. Doering, “NI myRIO-project essentials
E.Doering，“NImyRIO 项目要点

guide,” Natl. Instruments, vol. 129, pp. 55–59, 2014.
指南，“Natl.Instruments vol.129，pp.55–59,2014。

Y. Angal and A. Gade, “LabVIEW controlled robot for object handling using NI myRIO,” 2016 IEEE Int. Conf. Adv. Electron. Commun. Comput. Technol. ICAECCT 2016, pp. 167–171, 2017.
Y. Angal 和 A. Gade，“使用 NI myRIO 进行对象处理的 LabVIEW 控制机器人”，2016 年 IEEE 国际会议 Adv. Electron。 公社。计算。ICAECCT 2016，第 167-171 页，2017 年。

R. M. U. Manual, “Real-Time Module User
R.M.U.Manual，“实时模块用户”

Manual,” no. 322154, 2003.
手动，“不。 322154,2003 年。

F. Logic and T. User, “PID and Fuzzy Logic Toolkit User Manual,” Program, no. June, 2009.
F.Logic 和T.User，“PID和FuzzyLogic Toolkit 用户手册”，程序，编号。2009 年 6 月。

M. Davoudi and M. Davoudi, “Bayesian fuzzy logic-based estimation of electron cyclotron heating (ECH) power deposition in MHD control systems,” Fusion Eng. Des., vol. 119, pp. 35–41, 2017.
M. Davoudi 和 M. Davoudi，“MHD 控制系统中基于贝叶斯模糊逻辑的电子回旋加速器加热 （ECH） 功率沉积估计”，Fusion Eng. Des.，第 119 卷，第 35-41 页，2017 年。

Qazvin, Iran. His research interests are Image
伊朗，加兹温。他的研究兴趣是图像

processing and enhancement, Hardware implementation of controllers (fuzzy, adaptive, PID…) and Instrumentation.
处理和增强，控制器（模糊、自适应、PID 等）和仪表的硬件实现。