Pointcloud Stitching

Overview

Scalable, multicamera distributed system for realtime pointcloud stitching at IQR Lab. This program is currently designed to use D400 Series Intel RealSense depth cameras. Using the librealsense 2.0 SDK, depth frames are grabbed and pointclouds are computed on the edge, before sending the raw XYZRGB values to a central node over TCP sockets. The central program stitches the pointclouds together and displays a visualization with a PCL viewer.

host.py Script

This Python script is designed to run any arbitrary script on each of the edge nodes. We provide some useful scripts in edge_scripts/. Run python host.py -h to see usage instructions.

Installation

Different installation steps are required for installing on the edge nodes and the central node. The current instructions are tested on Ubuntu 22.04 LTS.

Central Node (Client)

  1. Follow the instructions to download and install the pcl from their website.
    • Ensure that your pcl version is 1.13 or later. If this version is not available in your package manager, you will need to download the source code and build it yourself.
  2. Clone this repository and cd into it.
  3. Build and compile the cetral node binaries.
mkdir -p build && cd build
cmake .. -DBUILD_CLIENT=true
make

Camera Edge Node (Server)

  1. Go to the IQR Lab librealsense guide and follow the instructions to install the librealsense 2.0 SDK.
  2. Ensure that your cmake version is 3.1 or later. If not, download and install a newer version from the CMake website
  3. There are two methods for obtaining the source
    1. Release - Clone this repository into ~/pointcloud_stitching
    2. FS Mount - This allows all edge servers to share one, live version of the source (resets on reboot)
      1. Create an SSH key on the edge server to the central computer
      2. Test the key and accept the fingerprint
      3. Install sshfs: sudo apt install sshfs
      4. Modify edge_scripts/mount.sh on the central computer with the correct user login, central computer hostname, absolute paths, and identity file
      5. Add this edge server's hostname to HOSTS
      6. Run python host.py edge_scripts/mount.sh
  4. Ensure that ~/pointcloud_stitching is the path of the local repo
  5. Run python host.py edge_scripts/build_edge.sh

Calibration

See calibration/README.md

Usage

Each RealSense is connected to an edge node, which are all accessible through ssh from the central node.

To start running, do the following:

  1. SSH into the edge nodes and run:

    ~/build/src/pcs-camera-optimized -s

    If the servers are setup correctly, each one should say Waiting for client...

  2. Then on the central node, run:

    build/src/pcs-multicamera-optimized -v

    This begins the pointcloud stitching (-v for visualizing the pointcloud).

    For more available options, run build/src/pcs-multicamera-optimized -h for help and an explanation of each option.