Skip to main content

darknet_ros_3d

Download the modified package for Team Galaxis​

Why do we need it?​

  • provides 3D bounding boxes for the detected traffic signs
    • in particular, the min- and max-coordinates of the detected signs which can be used for planning
  • see below for more details

Original package README​

Build Status

darknet_ros_3d​

Contents​

  • Overview
    • Previous steps
  • Quick start
    • Instalation
    • How it works
  • Nodes
    • darknet3d_node

Overview​

darknet_ros_3d provides you 3d bounding boxes of the objects contained in an objects list, where is specificated the 3d position of each object. Using a RGBD camera like Asus Xtion, Orbbec Astra or Intel Realsense and neuronal network darknet ros, objects can be detected and his position can be calculated.

In addition, there is a visual debugger tool based on visual markers that you can see with rviz

Image

Previous Steps​

The first we have to know is that darknet_ros_3d package have dependencies as follow:

  • rclcpp
  • rclcpp_lifecycle
  • darknet_ros_msgs
  • gb_visual_detection_3d_msgs
  • sensor_msgs
  • tf2
  • tf2_ros
  • tf2_sensor_msgs
  • visualization_msgs
  • eigen3_cmake_module
  • Eigen3

You can install Darknet Ros following this steps. NOTE: ros2 branch

Quick Start​

Instalation​

You must to clone gb_visual_detection_3d into src folder located in your workspace. Later, you have to compile it typing colcon build.

How It Works​

First of all, is necessary to run camera driver. Make sure that camera driver is publishing point cloud information.

Later, you must run darknet_ros and, if everything worked properly, you should see 2d bounding boxes in your screen. If not, you have a problem with darknet_ros package.

Now, you can run darknet_ros_3d typing ros2 launch darknet_ros_3d darknet_ros_3d.launch.py. If you want to change default parameters like topics it subscribe, you can change it in the configuration file located at ~/catkin_ws/src/gb_visual_detection_3d/darknet_ros_3d/darknet_ros_3d/config/. Default parameters are the following:

  • interested_classes: Classes you want to detect. It must be classes names than exists previously in darknet ros.

  • mininum_detection_threshold: Maximum distance range between any pixel of image and the center pixel of the image to be considered.

  • minimum_probability: Minimum object probability (provided by darknet_ros) to be considered.

  • darknet_ros_topic: topic where darknet_ros publicates it's bounding boxes. /darknet_ros/bounding_boxes.

  • point_cloud_topic: topic where point cloud is published from camera. By default: /camera/pointcloud. It is important that point cloud topic be of PointCloud2 type and it be depth_registered.

  • working_frame: frame that all measurements are based on. By default, camera_link. It is very important that if you want to change this frame, it has the same axes than camera_link, if you would want 3d coordinates in another axis, you must change it later (once 3d bounding box has been calculated).

Nodes​

darknet3d_node​

darknet3d_node provide bounding boxes. This bounding boxes are combinated with point cloud information to calculate (xmin, ymin, zmin) and (xmax, ymax, zmax) 3D coordinates.

Then, darknet_ros_3d publicates it's own bounding boxes array of BoundingBoxes3d type in /darknet_ros_3d/bounding_boxes_3d by default, which is an array of BoundingBox3d that contains the following information:

std_msgs/Header header
BoundingBox3d[] bounding_boxes

BoundingBox3d:

string object_name
float64 probability
float64 xmin
float64 ymin
float64 xmax
float64 ymax
float64 zmin
float64 zmax
  • object_name: Object name.
  • probability: Probability of certainty.
  • xmin: X coordinate in meters of left upper corner of bounding box.
  • xmax: X coordinate in meters of right lower corner of bounding box.
  • ymin: Y coordinate in meters of left upper corner of bounding box.
  • ymax: Y coordinate in meters of right lower corner of bounding box.
  • zmin: Z coordinate in meters of nearest pixel of bounding box.
  • zmax: Z coordinate in meters of the furthest pixel of bounding box.

An example of output is as follow:

header:
  stamp:
    sec: 1593723845
    nanosec: 430724839
  frame_id: camera_link
bounding_boxes:
- object_name: person
  probability: 0.7609682679176331
  xmin: 0.4506256580352783
  ymin: -0.3164764642715454
  xmax: 0.7936256527900696
  ymax: 0.11368180811405182
  zmin: -0.25958430767059326
  zmax: 0.10506562888622284

You can visualize the markers on rviz adding MarkerArray and subscribing to topic /darknet_ros_3d/markers.