基于临场感增强的果园机器人遥操作可视化系统研究
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:

江苏省重点研发计划项目(BE2017370)


Teleoperation Visualization System of Orchard Robot Based on Enhancing Telepresence
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    针对果园作业机器人使用单目相机进行遥操作时,仅用二维视频获取环境信息缺乏临场感的问题,设计了一套基于临场感增强的果园环境信息可视化系统,用于果园机器人遥操作。系统由计算服务器、云服务器、网络摄像头、激光雷达、嵌入式开发平台等组成。计算服务器采用T7920工作站,并在其上部署Tensorflow计算框架和Open3D点云算法库,计算服务器在接收到云服务器转发来的环境图像和点云数据后,分别对图像进行导航信息增强,对点云进行曲面重建;嵌入式开发平台可以收集来自于网络摄像头和激光雷达的原始数据,并上传至云服务器;在云服务器部署了以ZeroMQ为基础的消息中转程序和HTML5后台服务,提供跨互联网的消息通信服务和可移动的遥操作环境信息可视化服务。测试结果表明,部署在计算服务器的导航信息提取模型平均提取导航线时间86ms,提取导航线平均精度16°,均优于对比模型结果。点云重建算法可以有效建立场景轮廓,平均精度4.9cm,平均重建时间24ms。压缩图像传输及增强处理时延不超过230ms,点云的传输时延不超过400ms。各项参数可以满足遥操作机器人进行果园作业的基本要求,相比仅有单目相机的遥操作,临场感明显增强,可为果园机器人遥操作提供参考。

    Abstract:

    Aiming at the problem of lack of presence when the orchard robot used a monocular camera for remote operation, only using two-dimensional video to obtain environmental information lacked presence, a set of orchard environment information visualization system based on the enhanced sense of presence was designed. The system consisted of computing server, cloud server, network camera, LiDAR, embedded development platform, etc. The computing server adopted the T7920 workstation, and deployed the Tensorflow computing framework and the Open3D algorithm library of point cloud on it. After receiving the environmental image and point cloud data forwarded by the cloud server, the computing server enhanced the navigation information of the image, and surface reconstructed the point cloud. The embedded development platform could collect raw data from webcam and LiDAR, and uploaded them to cloud servers. A ZeroMQ-based message transfer program and HTML5 background service were deployed on the cloud server, providing cross-Internet message communication services and mobile teleoperation environment information visualization services. The test results showed that the average extraction time of the extraction model for navigation information deployed on the computing server was 86ms, and the average precision of the navigation line extraction was 16°, which were better than the results of the comparison model. The algorithm of point cloud reconstruction can effectively establish scene contours with an average accuracy of 4.9cm and an average reconstruction time of 24ms. The delay of compressed image transmission and enhancement processing did not exceed 230ms, and the transmission delay of point cloud did not exceed 400ms. The parameters could meet the basic requirements of the remote operation robot for orchard operation. The system significantly enhanced telepresence compared with that only with monocular cameras, which provided an effective reference for the remote operation of the orchard robot.

    参考文献
    相似文献
    引证文献
引用本文

王运东,周俊,孙经纬,王凯,江自真,张震.基于临场感增强的果园机器人遥操作可视化系统研究[J].农业机械学报,2023,54(3):22-31,78. WANG Yundong, ZHOU Jun, SUN Jingwei, WANG Kai, JIANG Zizhen, ZHANG Zhen. Teleoperation Visualization System of Orchard Robot Based on Enhancing Telepresence[J]. Transactions of the Chinese Society for Agricultural Machinery,2023,54(3):22-31,78.

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2022-06-17
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2023-03-10
  • 出版日期: