rk3588在ros中使用ros_rknn_yolo包

[xiaoqiang]

本文档将引导您如何在基于RK3588的设备上，使用ros_rknn_yolo软件包进行高效的对象检测。

1. 下载与编译

环境准备

确保您的RK3588设备已经安装了ROS环境。本教程默认您使用的是ROS1版本，具体的安装步骤请参考ros_rknn_yolo包内的readme文件。

克隆软件包

首先，切换到您的ROS工作空间src目录下，然后克隆ros_rknn_yolo软件包及其依赖的vision_msgs软件包：

cd ~/rk3588_ros_ws/src
git clone https://git.bwbot.org/publish/ros_rknn_yolo.git
git clone https://git.bwbot.org/publish/vision_msgs.git

编译软件包

返回到工作空间的根目录，进行编译：

#先安装系统依赖
sudo apt install python3-virtualenv
pip install opencv-python opencv-contrib-python
#开始编译
cd ~/rk3588_ros_ws/
catkin_make

编译完成后，需要运行ros_rknn_yolo包中的一个脚本以设置rknn2的运行环境：

roscd ros_rknn_yolo
sudo ./fix_rknn2_runtime

2. 启动Launch文件

ros_rknn_yolo包提供了Launch文件来启动节点和相关的配置。
在终端中运行以下命令以启动：

roslaunch ros_rknn_yolo xiaoqiang_yolo.launch

以上命令将启动rknn_yolo_node节点，并加载预定义的配置。有关节点参数和功能的详细信息，请参阅readme_cn.md文件。

3. 启动测试文件

为了验证ros_rknn_yolo包是否正常工作，可以运行包内提供的测试文件。
在新的终端窗口中，执行以下命令来启动测试节点：

roscd ros_rknn_yolo/test
python bus_do_yolo_srv.py

此测试脚本将向rknn_yolo_node节点发送一张图片，并显示检测结果。

您还可以自己运行其它test文件，比如同时推理两个视频

结语

至此，您已成功在RK3588设备上使用ros_rknn_yolo包进行了对象检测。根据您的具体需求，请进一步探索和调整ros_rknn_yolo包的配置和功能。

常见问题

a.这个包有两个分支，一个是master分支，还有一个是no-opencl分支，不想使用opencl加速的同学可以使用no-opencl分支。如果安装master分支时，提示无法安装python 依赖包，可能是opencl相关库没有安装好，可以先执行sudo ./fix_rknn2_runtime 。

b.这个包默认系统配置的python版本是3.10，其它版本的同学要将rknn_toolkit_lite2的安装包换成自己系统python版本的包，同时修改CMakeLists.txt中的88行。

[小助理]

这个文档提供了一个详细的步骤，指导如何在基于RK3588的系统上使用ROS和ros_rknn_yolo包来实现YOLO对象检测。以下是每个步骤的总结：

1. 环境准备：

   • 确保您的设备上已经安装了ROS1环境。
   • 在ROS工作空间的src目录下克隆ros_rknn_yolo和vision_msgs包。

2. 编译与环境设置：

   • 回到工作空间根目录并使用catkin_make编译。
   • 编译完成后，运行fix_rknn2_runtime脚本来设置运行环境。

3. 启动对象检测：

   • 使用roslaunch命令启动xiaoqiang_yolo.launch，它会启动rknn_yolo_node节点。

4. 验证功能：

   • 在另一个终端中，进入test子目录并运行bus_do_yolo_srv.py，这个测试脚本将发送图像数据给rknn_yolo_node并展示检测结果。

5. 进一步探索：

   • 可以根据需要调整ros_rknn_yolo包的配置和其他测试脚本。

请注意，这个过程假设您已经配置了必要的依赖，比如RKNPU2驱动、RKNN工具包和相应的模型。如果有任何问题，建议查阅readme_cn.md文件或联系软件包的开发者获取帮助。

[shlfan]

您好，我在运行到catkin_make时显示安装python依赖失败，查看CMakeError.log文件，显示如下图。通过测试，3588上有phtread。请问这是什么问题，该怎么解决？谢谢！

Performing C SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:
Change Dir: /home/ema/rk3588_ros_ws/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_48f6f/fast && /usr/bin/make -f CMakeFiles/cmTC_48f6f.dir/build.make CMakeFiles/cmTC_48f6f.dir/build
make[1]: 进入目录“/home/ema/rk3588_ros_ws/build/CMakeFiles/CMakeTmp”
Building C object CMakeFiles/cmTC_48f6f.dir/src.c.o
/usr/bin/cc   -DCMAKE_HAVE_LIBC_PTHREAD   -o CMakeFiles/cmTC_48f6f.dir/src.c.o   -c /home/ema/rk3588_ros_ws/build/CMakeFiles/CMakeTmp/src.c
Linking C executable cmTC_48f6f
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_48f6f.dir/link.txt --verbose=1
/usr/bin/cc  -DCMAKE_HAVE_LIBC_PTHREAD    -rdynamic CMakeFiles/cmTC_48f6f.dir/src.c.o  -o cmTC_48f6f 
/usr/bin/ld: CMakeFiles/cmTC_48f6f.dir/src.c.o: in function `main':
src.c:(.text+0x48): undefined reference to `pthread_create'
/usr/bin/ld: src.c:(.text+0x50): undefined reference to `pthread_detach'
/usr/bin/ld: src.c:(.text+0x5c): undefined reference to `pthread_join'
collect2: error: ld returned 1 exit status
make[1]: *** [CMakeFiles/cmTC_48f6f.dir/build.make:87：cmTC_48f6f] 错误 1
make[1]: 离开目录“/home/ema/rk3588_ros_ws/build/CMakeFiles/CMakeTmp”
make: *** [Makefile:121：cmTC_48f6f/fast] 错误 2


Source file was:
#include <pthread.h>

void* test_func(void* data)
{
  return data;
}

int main(void)
{
  pthread_t thread;
  pthread_create(&thread, NULL, test_func, NULL);
  pthread_detach(thread);
  pthread_join(thread, NULL);
  pthread_atfork(NULL, NULL, NULL);
  pthread_exit(NULL);

  return 0;
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /home/ema/rk3588_ros_ws/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_4bba2/fast && /usr/bin/make -f CMakeFiles/cmTC_4bba2.dir/build.make CMakeFiles/cmTC_4bba2.dir/build
make[1]: 进入目录“/home/ema/rk3588_ros_ws/build/CMakeFiles/CMakeTmp”
Building C object CMakeFiles/cmTC_4bba2.dir/CheckFunctionExists.c.o
/usr/bin/cc   -DCHECK_FUNCTION_EXISTS=pthread_create   -o CMakeFiles/cmTC_4bba2.dir/CheckFunctionExists.c.o   -c /usr/share/cmake-3.16/Modules/CheckFunctionExists.c
Linking C executable cmTC_4bba2
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_4bba2.dir/link.txt --verbose=1
/usr/bin/cc  -DCHECK_FUNCTION_EXISTS=pthread_create    -rdynamic CMakeFiles/cmTC_4bba2.dir/CheckFunctionExists.c.o  -o cmTC_4bba2  -lpthreads 
/usr/bin/ld: 找不到 -lpthreads
collect2: error: ld returned 1 exit status
make[1]: *** [CMakeFiles/cmTC_4bba2.dir/build.make:87：cmTC_4bba2] 错误 1
make[1]: 离开目录“/home/ema/rk3588_ros_ws/build/CMakeFiles/CMakeTmp”
make: *** [Makefile:121：cmTC_4bba2/fast] 错误 2

[xiaoqiang]

@shlfan 在 rk3588在ros中使用ros_rknn_yolo包 中说：

undefined reference to `pthread_join’

catkin_make时默认会编译整个工作空间中的包，你目前的错误是其它ros包有问题，可以使用下面命令只编译ros_rknn_yolo包

catkin_make -DCATKIN_WHITELIST_PACKAGES="vison_msgs;ros_rknn_yolo"

[shlfan]

@shlfan 在工作空间中只有这两个包，没有其他的包了，使用您给的代码最后得到的结果没有变化

ema@ema:~/rk3588_ros_ws$ catkin_make -DCATKIN_WHITELIST_PACKAGES="vison_msgs;ros_rknn_yolo"
Base path: /home/ema/rk3588_ros_ws
Source space: /home/ema/rk3588_ros_ws/src
Build space: /home/ema/rk3588_ros_ws/build
Devel space: /home/ema/rk3588_ros_ws/devel
Install space: /home/ema/rk3588_ros_ws/install
####
#### Running command: "cmake /home/ema/rk3588_ros_ws/src -DCATKIN_WHITELIST_PACKAGES=vison_msgs;ros_rknn_yolo -DCATKIN_DEVEL_PREFIX=/home/ema/rk3588_ros_ws/devel -DCMAKE_INSTALL_PREFIX=/home/ema/rk3588_ros_ws/install -G Unix Makefiles" in "/home/ema/rk3588_ros_ws/build"
####
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using CATKIN_DEVEL_PREFIX: /home/ema/rk3588_ros_ws/devel
-- Using CMAKE_PREFIX_PATH: /opt/ros/noetic
-- This workspace overlays: /opt/ros/noetic
-- Found PythonInterp: /home/ema/miniconda3/bin/python3 (found suitable version "3.8.18", minimum required is "3") 
-- Using PYTHON_EXECUTABLE: /home/ema/miniconda3/bin/python3
-- Using Debian Python package layout
-- Found PY_em: /usr/lib/python3/dist-packages/em.py  
-- Using empy: /usr/lib/python3/dist-packages/em.py
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/ema/rk3588_ros_ws/build/test_results
-- Forcing gtest/gmock from source, though one was otherwise available.
-- Found gtest sources under '/usr/src/googletest': gtests will be built
-- Found gmock sources under '/usr/src/googletest': gmock will be built
-- Found PythonInterp: /home/ema/miniconda3/bin/python3 (found version "3.8.18") 
-- Found Threads: TRUE  
-- Using Python nosetests: /usr/bin/nosetests3
-- catkin 0.8.10
-- BUILD_SHARED_LIBS is on
-- BUILD_SHARED_LIBS is on
-- Using CATKIN_WHITELIST_PACKAGES: vison_msgs;ros_rknn_yolo
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- ~~  traversing 1 packages in topological order:
-- ~~  - ros_rknn_yolo
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- +++ processing catkin package: 'ros_rknn_yolo'
-- ==> add_subdirectory(ros_rknn_yolo)
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Using these message generators: gencpp;geneus;genlisp;gennodejs;genpy
-- Found OpenCV: /usr (found version "4.2.0") 
CMake Error at /home/ema/rk3588_ros_ws/build/ros_rknn_yolo/cmake/ros_rknn_yolo-genmsg.cmake:3 (message):
  Could not find messages which
  '/home/ema/rk3588_ros_ws/src/ros_rknn_yolo/srv/DoYolo.srv' depends on.  Did
  you forget to specify generate_messages(DEPENDENCIES ...)?

  Cannot locate message [YoloResult] in package [vision_msgs] with paths
  [['/opt/ros/noetic/share/vision_msgs/cmake/../msg']]
Call Stack (most recent call first):
  /opt/ros/noetic/share/genmsg/cmake/genmsg-extras.cmake:307 (include)
  ros_rknn_yolo/CMakeLists.txt:33 (generate_messages)


-- Configuring incomplete, errors occurred!
See also "/home/ema/rk3588_ros_ws/build/CMakeFiles/CMakeOutput.log".
See also "/home/ema/rk3588_ros_ws/build/CMakeFiles/CMakeError.log".
Invoking "cmake" failed
ema@ema:~/rk3588_ros_ws$ catkin_make -DCATKIN_WHITELIST_PACKAGES="vision_msgs;ros_rknn_yolo"
Base path: /home/ema/rk3588_ros_ws
Source space: /home/ema/rk3588_ros_ws/src
Build space: /home/ema/rk3588_ros_ws/build
Devel space: /home/ema/rk3588_ros_ws/devel
Install space: /home/ema/rk3588_ros_ws/install
####
#### Running command: "cmake /home/ema/rk3588_ros_ws/src -DCATKIN_WHITELIST_PACKAGES=vision_msgs;ros_rknn_yolo -DCATKIN_DEVEL_PREFIX=/home/ema/rk3588_ros_ws/devel -DCMAKE_INSTALL_PREFIX=/home/ema/rk3588_ros_ws/install -G Unix Makefiles" in "/home/ema/rk3588_ros_ws/build"
####
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using CATKIN_DEVEL_PREFIX: /home/ema/rk3588_ros_ws/devel
-- Using CMAKE_PREFIX_PATH: /opt/ros/noetic
-- This workspace overlays: /opt/ros/noetic
-- Found PythonInterp: /home/ema/miniconda3/bin/python3 (found suitable version "3.8.18", minimum required is "3") 
-- Using PYTHON_EXECUTABLE: /home/ema/miniconda3/bin/python3
-- Using Debian Python package layout
-- Found PY_em: /usr/lib/python3/dist-packages/em.py  
-- Using empy: /usr/lib/python3/dist-packages/em.py
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/ema/rk3588_ros_ws/build/test_results
-- Forcing gtest/gmock from source, though one was otherwise available.
-- Found gtest sources under '/usr/src/googletest': gtests will be built
-- Found gmock sources under '/usr/src/googletest': gmock will be built
-- Found PythonInterp: /home/ema/miniconda3/bin/python3 (found version "3.8.18") 
-- Found Threads: TRUE  
-- Using Python nosetests: /usr/bin/nosetests3
-- catkin 0.8.10
-- BUILD_SHARED_LIBS is on
-- BUILD_SHARED_LIBS is on
-- Using CATKIN_WHITELIST_PACKAGES: vision_msgs;ros_rknn_yolo
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- ~~  traversing 2 packages in topological order:
-- ~~  - vision_msgs
-- ~~  - ros_rknn_yolo
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- +++ processing catkin package: 'vision_msgs'
-- ==> add_subdirectory(vision_msgs)
-- Using these message generators: gencpp;geneus;genlisp;gennodejs;genpy
-- vision_msgs: 14 messages, 0 services
-- +++ processing catkin package: 'ros_rknn_yolo'
-- ==> add_subdirectory(ros_rknn_yolo)
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Using these message generators: gencpp;geneus;genlisp;gennodejs;genpy
-- Found OpenCV: /usr (found version "4.2.0") 
-- ros_rknn_yolo: 0 messages, 1 services
git clone sahi success
created virtual environment CPython3.8.10.final.0-64 in 311ms
  creator CPython3Posix(dest=/home/ema/rk3588_ros_ws/src/ros_rknn_yolo/venv, clear=False, global=True)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, pkg_resources=latest, via=copy, app_data_dir=/home/ema/.local/share/virtualenv/seed-app-data/v1.0.1.debian.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Installing python dependencies of ros_rknn_yolo
/home/ema/rk3588_ros_ws/src/ros_rknn_yolo/venv/bin/pip /home/ema/rk3588_ros_ws/src/ros_rknn_yolo/requirements.txt
CMake Error at ros_rknn_yolo/CMakeLists.txt:73 (message):
  Install python dependencies failed


-- Configuring incomplete, errors occurred!
See also "/home/ema/rk3588_ros_ws/build/CMakeFiles/CMakeOutput.log".
See also "/home/ema/rk3588_ros_ws/build/CMakeFiles/CMakeError.log".
Invoking "cmake" failed

[xiaoqiang]

@shlfan 第一次运行错误，可以先只编译vision_msgs包

catkin_make -DCATKIN_WHITELIST_PACKAGES="vison_msgs"

第二次运行的错误是安装python虚拟环境的依赖包出了问题，手动执行CMakeLists.txt中的58行指令安装，看报什么错误，一般是网络问题。

还有一个注意点，这个包默认的python版本是3.10，你的是3.8版本，rknn_toolkit_lite2的安装包要自己换成3.8版本的，同时修改CMakeLists.txt中的88行。

[Natsuki]

您好，请问下，我再成功构建了ros_rknn_yolo包后，在按照教程到输入’roslaunch ros_rknn_yolo xiaoqiang_yolo.launch‘指令，并更改了default.yaml文件的模型路径后，出现了如下报错，能麻烦您看看这是怎么回事吗：

[xiaoqiang]

@natsuki 在 rk3588在ros中使用ros_rknn_yolo包 中说：

ros_rknn_yolo文件夹内有没有正确创建venv子文件夹，检查venv内的python虚拟环境是否正确创建了。
同时可以手动按行执行一下rknn_yolo_node.sh这个bash文件中的指令内容，看是哪一行出了问题。