Kai Liu

Kai Liu

Engineer @ UISEE, PhD @ Beijing Institute of Technology, Visiting scholar @ Ohio State University.

Using Apollo Algorithms in Your Normal ROS Package

6 minute read

This blog will introduce how the port_apollo repository converts an apollo module to a normal ROS package and uses Apollo algorithms in your own ros package in a native linux system (i.e. ubuntu 16.04 LTS) instead of docker environments.

The tested codes are based on the release apollo-v3.0.0.

The goal of this project is to convert a bazel project to a cmake project while without modifying the algorithm part of source codes in apollo and keeping the existing directory structure unchanged. But the #include directories in the source files somehow need to be adapted according to the structure of cmake projects. I am not going the split the header files and source files of apollo like typical cmake projects. I prefer keeping codes as it was to avoid copying files and moving directories. The rational behind this is that in this way I can easily checkout the contents of the module folder of new apollo release version and throw the existing CMakeLists.txt and package.xml to the updated codes and get a new working version with minimum effort. So the include directory won’t exist in every ROS package folder. But you can still include project header files like other normal ROS packages assuming the headers are there.

Structure of the port_apollo Repo

port_apollo
├── LICENSE
├── README.md
├── scripts
│   ├── build_pkg.sh  # handy build script for ros workspace
│   ├── docker_build_image.sh # docker image for CI, you can use it locally as well
│   ├── docker_compile_pkgs.sh # repo-build script in docker container
│   ├── Dockerfile   # base docker file to build a image: ubuntu 16.04 + ros kinetics
│   ├── docker_install_dependencies.sh # install dependencies in the docker container
│   ├── docker_run.sh  # run the docker for the CI test
│   ├── installer  # dependencies install scripts go here
│   └── tools  # tools scripts
└── src
    ├── catkin_simple  # dependencies, ros package
    ├── glog_catkin  # dependencies, ros package
    ├── cmake  # cmake module folder to hold FindProtobuf.cmake
    ├── common  # apollo module, ros package 
    ├── planning # apollo module, to be done
    ...
    └── other_module_name # apollo module, to be done

Converting Steps

Let’s create a new branch named feature_catkinizing_module to hold the all the changes we are going to make. Pick a module named module_name from planning,control, routing, perception,…, etc. to convert.

1. Create a working branch based on the master branch

git checkout master && git pull # pull the latest codes
git checkout -b feature_catkinizing_module # create a new branch locally
git push --set-upstream origin feature_catkinizing_module # push to the remote

2. Get the source codes of a module:

  • Checkout a module
    This repo holds a copy of the source codes of apollo-v3.0.0 in branch apollo_3_0_0. You can use the git command to checkout a folder from the branch apollo_3_0_0. 
    cd src && git checkout apollo_3_0_0 -- module_name # copy the module_name folder from apollo_3_0_0 to current working branch
  • Modify the #include directories in the header and source files in module_name directory
    All the original files include the modules/module_name in the #include lines. But in cmake projects, the prefix modules/module_name is not needed. My solution is searching all the .h and .cc files for modules/module_name/ and replacing it with an empty string "". In port_apollo repo, I provide a python script scripts/tools/content_hunter.py to do the work automatically. The usage is as below: 
    python scripts/tools/content_hunter.py [module1_name] [module2_name]

    The module names here can be common, planning, control, perception etc.

    # for example 
python scripts/tools/content_hunter.py common

    Note: The script only substitutes the string modules/module_name/. If the file include modules/other_module_name, you may need to remove it manually. Or improve the python script.

  • Create a test main file called module_name_tests.cc in the module_name folder.
    This file will call the test cases you are going to add in the CMakeLists.txt file. 
      #include <gtest/gtest.h>
  int main(int argc, char **argv) {
    ::testing::InitGoogleTest(&argc, argv);
    return RUN_ALL_TESTS();
  }

  • Add CMakeLists.txt

      cmake_minimum_required(VERSION 3.0)
  project(module_name VERSION 0.0.1 LANGUAGES CXX)
  # enable c++11　feature
  set(CMAKE_CXX_STANDARD 11)
  #  enable tests
  set(CATKIN_ENABLE_TESTING ON)
  # for protobuf to generated interfacing files
  list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../cmake")
  ###############
  #### set the catkin dependend packages
  set(PKG_DEPS
      glog_catkin
      roscpp)
  find_package(catkin REQUIRED COMPONENTS ${PKG_DEPS})
  find_package(Protobuf REQUIRED)
  # set the .proto definition files  
  set(PROTOS
        path/to/file.proto # relative to module_name folder   
  )
  # call the protobuf command to generate the header and source files
  PROTOBUF_GENERATE_CPP(PROTO_SRCS PROTO_HDRS ${PROTOS})

    Note: I write a custom cmake module called FindProtobuf.cmake to find the protobuf library and related variables. It depends on pkg_check_modules of the pkgconfig and protobuf.pc to find the right version and return below variables

      PROTOBUF_INCLUDE_DIRS
  PROTOBUF_LIBRARIES
  RPOTOBUF_FOUND

    It also provides a cutom cmake command PROTOBUF_GENERATE_CPP to generate the header and source files for the .proto definition files before compiling the library. Currently, the script does not install .proto files to the shared folder like catkin_ws/devel/share or catkin_ws/install/share. The generated files will mirror the .proto directory structure in module_name folder. So if you need to use the generated header file of some .proto, just include the directory_to_proto file but replace the extension .proto with the .pb.h. For example, #include "common/configs/proto/vehicle_config.pb.h" for common/configs/proto/vehicle_config.proto. Of course, in the same package, you don’t need the module name common.

      # global include directories
  include_directories(
      ${catkin_INCLUDE_DIRS}
      ${PROTOBUF_INCLUDE_DIRS}
  )
  # use catkin_package command to expose the header files and the same directory structure to other catkin packages
  catkin_package(
      ## export these folder sturcture in the package root folder to other catkin packages
      INCLUDE_DIRS ${CATKIN_DEVEL_PREFIX}/include
      LIBRARIES ${PROJECT_NAME}  # we are going to wrap all the algorithms to one library named by the module_name
      CATKIN_DEPENDS ${PKG_DEPS})
  # set the source files for the library, you can select the source files you want to test in the module. Be careful with the dependencies. You can check the bazel build files for the dependency tree. You may need other module to be a catkin package first. In that case, convert other module first. Then add other modules as catkin dependencies to PKG_DEPS
  set(SOURCES
      path/to/source1.cc
      path/to/source2.cc)
  # add library target
  add_library(${PROJECT_NAME}
      ${SOURCES}
      ${PROTO_HDRS}
      ${PROTO_SRCS})
  # set the include directory only for the target
  target_include_directories(${PROJECT_NAME}
      PUBLIC
        $<BUILD_INTERFACE:${CATKIN_DEVEL_PREFIX}/include>
        $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
      PRIVATE
        $<BUILD_INTERFACE:${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}>
      )
  # link the library
  target_link_libraries(${PROJECT_NAME}
      ${catkin_LIBRARIES}
      ${PROTOBUF_LIBRARIES}
      )
  if (CATKIN_ENABLE_TESTING)
      catkin_add_gtest(${PROJECT_NAME}_tests
          module_name_tests.cc
          path/to/test1.cc
          path/to/test2.cc)

      target_include_directories(${PROJECT_NAME}_tests
        PUBLIC
          $<BUILD_INTERFACE:${CATKIN_DEVEL_PREFIX}/include${PROJECT_NAME}>
          $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
        PRIVATE
          $<BUILD_INTERFACE:${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}>
          )
      target_link_libraries(${PROJECT_NAME}_tests
        ${catkin_LIBRARIES}
        ${PROJECT_NAME})

  endif (CATKIN_ENABLE_TESTING)
    
  #############
  ## Install ##
  #############
  ## install the headers and directories
  ## don't miss any subfolders of the root, or you can't include the headers
  ## from other ros packages
  # take the `common` module for example
  set(DIRS proto math util time vehicle_state data configs adapters status transform_listener monitor_log kv_db filters)
  # install all the headers to devel space since we export ${CATKIN_DEVEL_PREFIX}/include/
  # to catkin package instead of the include folder in the root
  install(DIRECTORY ${DIRS}
            DESTINATION ${CATKIN_DEVEL_PREFIX}/include/${PROJECT_NAME}
            FILES_MATCHING PATTERN "*.h"
            PATTERN ".svn" EXCLUDE
    )
  # install the header files in the module_name root folder
  install(FILES log.h macro.h
            DESTINATION ${CATKIN_DEVEL_PREFIX}/include/${PROJECT_NAME}
  )
  # install all the headers to the install space, mirror the structure
  # install the generated headers
  install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}
            DESTINATION ${CATKIN_GLOBAL_INCLUDE_DESTINATION}
            FILES_MATCHING PATTERN "*.h"
            PATTERN ".svn" EXCLUDE
  )
  # install all the headers to install space
  install(DIRECTORY ${DIRS}
            DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
            FILES_MATCHING PATTERN "*.h"
            PATTERN ".svn" EXCLUDE
  )
  # install header in the module_name root folder to install space
  install(FILES log.h macro.h
            DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
  )

  • Add package.xml, should match the ros packages defined by PKG_DEPS in CMakeLists.txt.

      <package format="2">
    <name>module_name</name>
    <description>apollo module_name module</description>
    <maintainer email="abc@abc.com">abc</maintainer>
    <license>Apache 2.0</license>
    <version>3.0.0</version>
    <buildtool_depend>catkin</buildtool_depend>
    <depend>glog_catkin</depend>
    <depend>roscpp</depend>
  </package>

4. Compile and Test, in port_apollo

  • Build and run tests

      catkin build
  catkin build --make-args tests
  ./devel/lib/module_name/module_name

5. Call the algorithms from the module_name you just converted in your catkin package my_package

  • CMakeLists.txt

      find_package(catkin REQUIRED COMPONENTS module_name)
  catkin_package(
      CATKIN_DEPENDS module_name
  )

  • package.xml

      <package format="2">
    <name>my_package</name>
    <description>my_package</description>
    <maintainer email="yu.zhang.bit@gmail.com">Yu Zhang</maintainer>
    <license>Apache 2.0</license>
    <version>0.0.1</version>
    <buildtool_depend>catkin</buildtool_depend>
    <depend>module_name</depend>
  </package>

  • Include headers from module_name package in the source files of your own package. Just follow the directory structure in module_name package. You can also use clion IDE to help you find the headers automatically.

      #include <module_name/subfolder/some_header.h>

A working example could be found in package common.

Pull requests for port_apollo are welcome!

Leave a Comment

You May Also Enjoy

Nvidia Driver and Cuda9.0 Installation

8 minute read

Tested hardware and OS configuration: OS: Ubuntu 16.04 LTS NVIDIA Graphic Card: Quadro M1000M Cuda Version: 9.0 Graphic Card Driver Version: 410.xx ...

Use apt-get behind a socks5 proxy

less than 1 minute read

Nothing would be more frustrating than to install apt packages from sources outside of China, especially from PPA. If you try to update the c++ compiler to g...

Several Ways of Writing a ROS Node

5 minute read

The design of nodes varies according to the requirements of different applications. Selecting the right node pattern is pretty important for achieving certai...