Pointcloud Stitching for IQR Lab

Overview

Scalable, multicamera distributed system for realtime pointcloud stitching in IQR Lab. This program is currently designed to use the D400 Series Intel RealSense depth cameras. Using the librealsense 2.0 SDK, depth frames are grabbed and pointclouds are computed on the edge, before sending the raw XYZRGB values to a central computer over a TCP sockets. The central program stitches the pointclouds together and displays it a viewer using PCL libraries.

host.py Script

This Python script is designed to run any arbitrary script on each of the edge servers. We provide some useful scripts in edge_scripts/. Run python host.py -h to see usage instructions.

Note: host.py follows the iqr-vision-i.local hostnaming convention.

Installation

Different steps of installation are required for installing the realsense camera servers versus the central computing system. The current instructions are for running on Ubuntu 22.04 LTS.

Central Computer

  1. Follow the instructions to download and install the pcl from their website.
    • Ensure that your pcl version is 1.13 or later. If this version is not available in your package manager, you will need to download the source code and build it yourself.
  2. Clone this repository and cd into it.
  3. Build and compile the central computing system binaries.
mkdir -p build && cd build
cmake .. -DBUILD_CLIENT=true
make

Camera Edge Server

  1. Go to the IQR Lab librealsense guide and follow the instructions to install the librealsense 2.0 SDK.
  2. Ensure that your cmake version is 3.1 or later. If not, download and install a newer version from the CMake website
  3. There are two methods for obtaining the source
    1. Release - Clone this repository into ~/pointcloud_stitching
    2. FS Mount - This allows all edge servers to share one, live version of the source
      1. Create an SSH key on the edge server to the central computer
      2. Test the key and accept the fingerprint
      3. Install sshfs: sudo apt install sshfs
      4. Modify edge_scripts/mount.sh on the central computer with the correct user login, central computer hostname, absolute paths, and identity file
      5. Add this edge server's hostname to HOSTS
      6. Run python host.py edge_scripts/mount.sh
      7. To set this FS mount as permanent, please follow these instructions
  4. Ensure that ~/pointcloud_stitching is the path of the local repo
  5. Run python host.py edge_scripts/build_edge.sh

Usage

Each RealSense is connected to an edge computer, which are all accessible through ssh from the central computer.

To start running, do the following:

  1. SSH to each edge computer and run:

    ~/build/src/pcs-camera-optimized -s

    If the servers are setup correctly, each one should say Waiting for client...

  2. Then on the central computer, run:

    build/src/pcs-multicamera-optimized -v

    This begins the pointcloud stitching (-v for visualizing the pointcloud).

    For more available options, run build/src/pcs-multicamera-optimized -h for help and an explanation of each option.