Depth sensors such as OPT8241 CDK from Texas Instruments enable a wide array of new applications, ranging from people counting to gesture control to robotics and other dimensioning applications. 3D sensing together with a WiFi-enabled embedded system, will enable powerful IoT applications. This blog post describes how to integrate OPT8241 CDK with Raspberry Pi 3.
Download Ubuntu MATE
Download UbuntuMATE 16.04 LTS image into your Ubuntu Linux box and unzip the *.img.xz file:
xz -d filename.img.xz
where filename.img.xz
is the downloaded image.
Program Micro SD Card
The next step is program the image into the micro SD card. Insert the micro SD card into your Ubuntu Linux PC and check the SD card’s assigned device name this way:
dmesg | tail
Usually you should see something like mmcblk0
or sdc
. Remember the device name. Eject or unmount the micro SD card, and then program the micro SD card using this command:
dd if=filename.img of=/dev/mmblk0 bs=1M
Replace filename
and mmblk0
with your own filename and device name.
Check Disk Partitions
The Ubuntu Mate image downloaded may not be configured to use of the full storage capacity of your micro SD card. To ensure the full storage capacity is used, run gparted
:
sudo apt-get install gparted sudo gparted
Click on the Linux partition (not the boot partition), and select Check
from the menu, and option will be provided to resize the selected partition to the full capacity.
Booting the Image
Once programming is completed, insert the micro SD card into RPi3 and power it up with HDMI and USB keyboard and mouse connected. Answer a series of questions and enable WiFi connection. WiFi may become disconnected, but no worries, just reboot. When booting and general OS setup is completed, login and create a subdirectory Software
at the home directory to keep all the new software we will download:
cd mkdir Software
Setup Swapfile
Create a swap file to supplement the limited 1GB RAM available on RPi3. A swap file is space on the storage device used as virtual memory, useful in compiling massive projects.
First, check that there is no active swap by entering the command:
free -h
Create a 2 GB swap file by the following commands:
sudo fallocate -l 2G /swapfile ls -lh /swapfile # Check that the file is indeed 2GB sudo chmod 600 /swapfile # Make the file only accessible to root sudo mkswap /swapfile # Identifies the file as swap space sudo swapon /swapfile # Enables swap file sudo swapon --show # Verifies that the swap file is working
To make the swap permanent, add the below line to /etc/fstab
and reboot:
/swapfile none swap sw 0 0
Setup Prerequisites
To prepare the software development on RPi3 we need setup some prerequisite tools:
sudo apt-get update sudo apt-get install git build-essential linux-libc-dev sudo apt-get install cmake cmake-gui swig3.0 sudo apt-get install libusb-1.0-0-dev libusb-dev libudev-dev sudo apt-get install mpi-default-dev openmpi-bin openmpi-common sudo apt-get install libflann1.8 libflann-dev sudo apt-get install libeigen3-dev sudo apt-get install libboost-all-dev sudo apt-get install libvtk5.10-qt4 libvtk5.10 libvtk5-dev sudo apt-get install libqhull* libgtest-dev sudo apt-get install freeglut3-dev pkg-config sudo apt-get install libxmu-dev libxi-dev sudo apt-get install mono-complete sudo apt-get install qt-sdk openjdk-8-jdk openjdk-8-jre
Build Point Cloud Library
Point Cloud Library is needed by Voxel SDK, and can be obtained by:
sudo add-apt-repository ppa:v-launchpad-jochen-sprickerhof-de/pcl sudo apt-get update git clone https://github.com/PointCloudLibrary/pcl.git
Now you should find pcl
in the ~/Software
directory. Create a release directory and follow the cmake
build process:
mkdir release cd release cmake -DCMAKE_BUILD_TYPE=None -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_GPU=ON -DBUILD_apps=ON -DBUILD_examples=ON -DCMAKE_INSTALL_PREFIX=/usr .. make
The make
will take over 7 hours. Once the build finishes, install it by:
sudo make install
Build Voxel SDK
In the same Software
directory, clone Voxel SDK:
cd ~/Software git clone https://github.com/3dtof/voxelsdk.git
Go the voxelsdk
directory and edit the top level CMakeList.txt
with the below change:
# set(ARCH ${CMAKE_SYSTEM_PROCESSOR}) set(ARCH armhf)
and
#add_definitions(-msse2 -pthread -std=c++11 -fPIC -ffast-math) add_definitions(-pthread -std=c++11 -fPIC -ffast-math -fpermissive)
Also the CMakeList.txt
in the VoxelPCL
subdirectory change the “wily” option by removing “1.7” from all the libpcl-*1.7
, then change “wily” to “xenial”. (Note: Ubuntu 16.04 codename is “xenial”).
Create a build directory and run cmake-gui
:
mkdir build cd build cmake-gui ..
Change CMAKE_INSTALL_PREFIX
to /usr
, and make all Python to use Python 2.7 include and library instead of Python 3.5m. After you made these changes, click Configure and then Generate to create the makefile.
To build Voxel SDK enter:
make
During build you may get an error about duplicated “standard container” macro definition between one in swig3.0
and one in Voxel SDK–resolve it by commenting out the problematic macro in the Voxel SDK SWIG directory.
After a successful build, install Voxel SDK by:
sudo make install
This will install need header files to /usr/include/voxel-0.6.5
and libraries to /usr/lib/voxel
. Create a symbolic link as follow:
sudo ln -s /usr/include/voxel-0.6.5 /usr/include/voxel
This is required before building demo applications below.
Build DemoApplications
Now we will test the installed Voxel SDK with demo applications available on GitHub:
cd ~/Software git clone https://github.com/3dtof/DemoApplications.git cd DemoApplications/TinTin/people_tracking
Now remove the -msse2
flag from CMakeLists.txt
.
#add_definitions(-msse2 -pthread -std=c++11 -fPIC -ffast-math) add_definitions(-pthread -std=c++11 -fPIC -ffast-math -fpermissive)
If your machine does not have OpenCV installed, install it this way:
sudo apt-get install libopencv-dev
Now we are ready to build the demo application:
mkdir build cd build cmake .. make ./PeopleTracking
and you should see the demo running. If the demo is dropping frames or having the “slow forward pipeline” warnings, reduce the frame dimension by editing the main()
inside the source file PeopleTracking.cpp
so it looks like this:
//Horus eye(80, 60) Horus eye(160, 120) //Horus eye(320, 240)
Now run make
again and everything should be working smoothly.
Below is a video of the setup running one of the applications in the DemoApplications
repository.
Hi Larry, is this possible to use sensor_msgs::PointCloud2 with this camera. If yes, why there is no support on Voxel SDK. Many thanks in advance for your kind reply!
Voxel SDK AFAIK is not specific to ROS. You will need to perform pixel copy from the native depth map to the PointCloud2 structure.
Hi Larry,
I have PCL 1.9.1 and Voxel SDK 0.6.11. I am trying to connect OPT8241 with ROS however I couldn’t make it yet. I have checked TI support forum however they confirm that there is no API for ros interfacing of this camera. Do you have any suggestion for me so after interfacing with ROS I can use it in rviz and create a 3D mapping? I’ll appreciate if you could answer for a beginner level one.
Thanks a lot!
Best regards.
Hi Dee,
Have you find a way to make it with ROS?
Cheers!
Hallo,
I am running Ubuntu Mate as suggested by TI, but on my raspberry Pi 3B+, 64GB Card it fails when I want to compile PCL. It tells me Virtual Memory is exhaustetd (at about 61%), but I have enabled a swapfile and already tried to increase that to 4GB, but I got stuck at the same place (PCL Surface).
Did you have any similar problem. Well, I needed to configure the makefile via ccmake .. as the command given by TI and above here did produce error (GPU Module configuration). Do you have any suggestions? Probably with the config of the makefile, but actually I am unsure…
Hello, is it required to use Python 2.7 instead of Python 3.5?
Python 3 and 2 are quite different. Python 2.7 is required at the time my blog was written.
Hi,,
Really Thanks for the great steps provided to work with PCL …
I am using Occipital Structure sensor with Raspberry pi 3, I was able to build properly. but runnig a demo application shows below output message
./PeopleTracking
Cannot connect
I found that, this _sys.scan(); function returns 0 and hence not able to connect.
i coundn’t trace completely the issue, kindly help me to resolve the issue ..
Thanks
For Occipital sensor you will need to use Openni2 instead of voxel sdk.
In your CMakeLists.txt you need something like that
include_directories(/usr/include/voxel-0.6.11/ti3dtof)
include_directories(/usr/include/voxel-0.6.11/ ${EIGEN_INCLUDE_DIRS} )
FIND_PACKAGE(Voxel 0.1.0 REQUIRED)
find_package(OpenCV REQUIRED)
set(THREADS_PREFER_PTHREAD_FLAG ON)
find_package (Threads REQUIRED )
TARGET_LINK_LIBRARIES(myapp ${OpenCV_LIBS} ${VOXEL_LIBRARIES} ${Boost_LIBRARIES} )
Thanks Larry..
Hey SATHEESH RAMASAMY i am trying to do the same thing and i am having a really hard time.Did you manage to make it finally?And if yes would you please help me to resolve the problem?
Hi Larry, your blog is very interesting for me. I have learned a lot about the R PI3 and the OPT8241. I developed an application but I wanted to know how to capture the error when the camera is disconnected. Can you send a suggestion?
When camera is disconnected you will not get any valid frames, so a simple timeout mechanism should work.
Hi Larry,
and thanks a lot for your work.
I got troubles running the last version of Jessie on a 16Gb SD card. I was able to install all packages, and when i do the make from PCL, it took to my computer 4 day to compile for finish with a C++ internal compiler error:
sample_consensus/CMakeFiles/pcl_sample_consensus.dir/build.make:134
make[2]: *** [sample_consensus/CMakeFiles/pcl_sample_consensus.dir/src/sac_model_cylinder.cpp.o] error 4
CMakeFiles/Makefile2:339
sample_consensus/CMakeFiles/pcl_sample_consensus.dir/all
make[1]: *** [sample_consensus/CMakeFiles/pcl_sample_consensus.dir/all] error 2
Makefile:160
make: *** [all] error 2
I used a 32GB SSD card and you also need to setup swapfile, otherwise the build will run out of memory and crash.
I am not sure if I have already replied, but you may need to setup a swapfile.
Hello Larry,
I’m having a bit problem when i try to run cmake-gui .. in “build” file. I made build folder like this “/Software/voxelsdk/build” and after typing cmake-gui .. i get this problem “libEGL warning: DRI2: failed to authenticate” and after that a pop-up comes up and tells me to select a source code. I tried to make build folder in different directories but the error is the same. i’m kinda new to this so i don’t know where i made the mistake. Until this step everything worked out smoothly.
It would be a huge help if you can help. Thanks.
Just to be clear, you need to make it with ‘mkdir build’ then cd into it then cmake-gui .. from there.
Yes i did it like this. https://prnt.sc/i144uq
Similary i did the same thing in /Software/pcl/release while building point could library via cmake and there was no error.
Finally solved the problems while building from source! I had to upgrade to Vtk6 and install OpenNI since this time it was giving me the error at PCL building too. Due to updated version of PCL(v1.8.1) and Voxel(v0.6.11) the prerequisites must be updated too, i guess.
Just for your information.
Hello Larry,
I’m having a bit problem when i try to run cmake-gui .. in “build” file. I made build folder like this “/Software/voxelsdk/build” and after typing cmake-gui .. i get this problem “libEGL warning: DRI2: failed to authenticate” and after that a pop-up comes up and tells me to select a source code. i’m kinda new to this so i don’t know where i made the mistake.
I would be a huge help if you can help. Thanks.
Hello KAGAN
Could you please explain more about how you solved it? I met the same problem as you. Thank you very much.
Hi chen,
Try to update prerequisites such as Vtk6 and OpenNI. PCL and Voxel are updated so the required dependencies must be updated too.
Hey Larry, I’m stuck at making the demo application. I’m getting missing file error and if I’m following the steps correctly. This demoapplication should be a subfolder of Software, not in voxelsdk or anywhere else. From there, I should make the build file inside the people_tracking folder and build the file from there, right?
Cheers!
You should be able to place demo application repo anywhere.
I changed the directory to ‘Software/pcl/build’. Then I issued cmake and make.
After cmake it looks like this: https://prnt.sc/fr1cpu
When I type ‘make’ afterwards there is either an internal compiler error OR an error like this:https://prnt.sc/fr1d0d
Where is my mistake 🙁
From your command line prompt it looks like you launched cmake from ‘Software/release’ but the build files are written to ‘Software/pcl/release’. Please first reconcile that issue.
For your info, using the latest Raspian Jessie Pixel, I was not able to install the libvtk stuff). BTW, there are some lines around QT, Mono and JDK. Do we really need these for getting the Kinect to run? They need another GB on flash…
You can try without them and see if everything work. I am using a 32GB SSD.
Hi there!
I was just trying to go through all these steps when two errors occured:
1st: Download failed: http/ppa.launchpad.net/v-launchpad-jochen-sprickerhof-de/pcl/ubuntu/dists/xenial/main/binary-armhf/Packager
–> ignored or older ones used.
2nd error came up when I enter the cmake -DCMAKE (…)
“The source directory “/home/ralph” does not appear to contain CMakeLists.txt.”
I am anything but pro at programming but I somehow have to make this work. Is there a connection between these errors or do you have any suggestion on how to make this work? 🙂
Of course, you can contact me directly via email.
Best regards,
Ralph
Are you running Raspbian or Ubuntu Mate?
I am running UbuntuMATE.
Also, I re-entered the cmake command today (without the “..” at the end) and was able to start the make process. Though, after 3-4% the following errors occur:
c++: internal compiler error: Killed (program cc1plus)
…
full error can be found here (screenshot): https://prnt.sc/fop2yd
Thanks for your help!
Ralph,
cmake works needs to find CMakeLists.txt, which sets in the root directory containing the source. To build the source, create a ‘build’ directory under the source directory, then ‘cd’ into it. Then issue the “cmake .. ” command. That’s the correct way build.