Mario Badr

Creating a Header-Only Library with CMake

It seems like header-only libraries are all the rage in C++ these days. In this post, we'll see a useful way to organize your header only library and generate a CMake target that others can "link" to. We will make use of CMake's INTERFACE library feature to do this:

An INTERFACE library target does not directly create build output, though it may have properties set on it and it may be installed, exported and imported.

Hopefully by the end of this post you will understand why this is so useful!

The Directory Structure

There are a few options here, but in general it makes sense to have an include directory for all the header files in your library. You may then place your headers directly in the include directory, as in cxxopts. Or you may choose to create an additional subdirectory with the same name as your library's namespace, as in type_safe. So for a project that links with your library, a developer can #include in one of two ways:

#include <cxxopts.hpp>
#include <type_safe/strong_typedef.hpp>

Examples and test files should go into their own directory and not pollute the include directory. This provides two benefits:

  1. It is easy to use CMake's target_include_directories feature
  2. It is easy to install the include directory into a destination

Of course, this is not set in stone. For example, nlohmann's json library places its header files in a src directory. But tests and benchmarks are still relegated to their own directories.

Creating a Header-Only CMake Target

Creating an interface library in CMake is very straightforward:

add_library(my-library-name INTERFACE)

By specifying INTERFACE as the second parameter to add_library, we are no longer allowed to provide source files since the library is not meant to generate any build output. However, if you would like your library to be IDE friendly, then you can specify the source files using target_sources, which is done by the type_safe library. target_sources was added in CMake 3.1 and allows headers that are not part of the executable target to be displayed in IDEs. (There are also other benefits to target_sources).

Once you've created a target for your header-only library, you should at the very least specify its target_include_directories. If you followed the directory structure above, then it is as simple as:

target_include_directories(my-library INTERFACE include/)

We specify INTERFACE again here so that future targets that link with our library will inherit our library's include directory. This is extremely convenient and is the main reason we are creating an INTERFACE library. We can continue to use INTERFACE when using target_link_libraries, target_compile_definitions, etc. so that we can propagate dependencies to targets that link with our library. In other words, the only requirement another developer has in using your library is to:

target_link_library(another-users-target my-library)

Doing so will ensure that any INTERFACE-marked include directories, compile definitions, other linked libraries, etc. are also added to another-users-target.

INTERFACE versus PUBLIC versus PRIVATE

It may be handy to understand how INTERFACE, PUBLIC, and PRIVATE behave. If PRIVATE is specified for a certain option/property, then that option/property will only impact the current target. If PUBLIC is specified, then the option/property impacts both the current target and any others that link to it. If INTERFACE is specified, then the option/property does not impact the current target but will propagate to other targets that link to it.

This makes INTERFACE very useful for header-only libraries that aren't compiled (on their own), but rely on other libraries they need to link to. Using target_link_library on our header-only library with INTERFACE specified means that the executable that eventually gets compiled with our library will have the appropriate link flags.

Using clang-format to Enforce Style

Everyone has their own coding style. Conforming to one particular style or another can be difficult, but with clang-format it doesn't have to be. Given a file (or collection of files), clang-format will modify it so that it meets certain style requirements. Several pre-configured styles already exist, such as llvm, Google, Chromium, Mozilla, and Webkit. For example, suppose we have the main.cpp file:

#include <iostream>

int main() {
std::cout << "Hello world\n";
return 0;
}

We can format the file according to Mozilla's styleguide with:

clang-format -i -style=Mozilla main.cpp

And end up with:

#include <iostream>

int
main()
{
  std::cout << "Hello world\n";
  return 0;
}

If you don't like any of the pre-configured styles, you can create a .clang-format file with your own rules. Simply replace Mozilla with file and clang-format will look for a .clang-format file in your current directory to use. Check out this amazing clang-format configurator to help define your own style.

Using clang-format with CMake

If you are using CMake to build your project, then it probably makes sense to have it handle clang-format for you. The first thing to do is to check if the clang-format executable exists:

find_program(
  CLANG_FORMAT_EXE
  NAMES "clang-format"
  DOC "Path to clang-format executable"
  )
if(NOT CLANG_FORMAT_EXE)
  message(STATUS "clang-format not found.")
else()
  message(STATUS "clang-format found: ${CLANG_FORMAT_EXE}")
  set(DO_CLANG_FORMAT "${CLANG_FORMAT_EXE}" "-i -style=file")
endif()

Using DO_CLANG_FORMAT we can simply pass in the files that need to be formatted. Typically a CMake project will have a list with all the files that are part of a given target:

set(
  MY_PROJECT_SOURCE_FILES
  include/hello_world.hpp
  src/main.cpp
  )

Unfortunately passing these files straight to our DO_CLANG_FORMAT won't work because the path to the files is relative. We need to prefix each file with the full path in order to get it to work. Searching around stackoverflow I came across a handy prepend function to do just that. The first argument passed to the function will contain the result, the second argument is the prefix, and the last argument (not named in the function) is a list of files to be prefixed.

function(prepend var prefix)
  set(listVar "")

  foreach(f ${ARGN})
    list(APPEND listVar "${prefix}/${f}")
  endforeach()

  set(${var} "${listVar}" PARENT_SCOPE)
endfunction()

if(CLANG_FORMAT_EXE)
  prepend(FILES_TO_FORMAT ${CMAKE_CURRENT_SOURCE_DIR} ${MY_PROJECT_SOURCE_FILES})

  add_custom_target(
    clang-format-project-files
    COMMAND ${CLANG_FORMAT_EXE} -i -style=file ${FILES_TO_FORMAT}
  )
endif()

CMake will now generate a custom target that can be executed to format the files. If you use makefiles, it can be invoked easily with:

make clang-format-project-files

And there you have it - free formatting!

Using clang-tidy with CMake 3.6

In CMake 3.6 a new property was introduced to support clang-tidy. clang-tidy can perform static analysis of your code and provide you with additional information to compiler warnings. In this post we'll take a look at how to make use of the new CMake property to add clang-tidy to our projects. If you're interested in how to use the different features of clang-tidy, I refer you to Google :).

Finding clang-tidy

clang-tidy may not be available on all systems, so in typical cmake fashion we first attempt to find the executable. If the executable is available, I setup a convenient CMake list (DO_CLANG_TIDY) that will help when using the CMake property later. In this case, I have enabled all checks with *, and then subsequently disabled checks that are currently in alpha (using the * wildcard). After you run clang-tidy with your application, you will likely disable additional checks. In that case, you will need to separate removed checks with a comma and ensure they are prefixed with a -.

find_program(
  CLANG_TIDY_EXE
  NAMES "clang-tidy"
  DOC "Path to clang-tidy executable"
  )
if(NOT CLANG_TIDY_EXE)
  message(STATUS "clang-tidy not found.")
else()
  message(STATUS "clang-tidy found: ${CLANG_TIDY_EXE}")
  set(DO_CLANG_TIDY "${CLANG_TIDY_EXE}" "-checks=*,-clang-analyzer-alpha.*")
endif()

Setting the property

With DO_CLANG_TIDY, we can now set the CXX_CLANG_TIDY (assuming you are using C++) property for all targets we want to analyze.

set_target_properties(
  my-target-name PROPERTIES
  CXX_STANDARD 14
  CXX_STANDARD_REQUIRED ON
  COMPILE_FLAGS "${WARNING_FLAGS}"
)

if(CLANG_TIDY_EXE)
  set_target_properties(
    my-target-name PROPERTIES
    CXX_CLANG_TIDY "${DO_CLANG_TIDY}"
  )
endif()

The above will enable C++14 and setup some custom warning flags. If clang-tidy is available, then it will run DO_CLANG_TIDY whenever we build the target. That's basically all there is to it!

Ubuntu 14.04 Disk Images for gem5

There is an excellent post by Jason on creating disk images for gem5. Unfortunately I ran into several issues while trying to reproduce his work. The first set of issues stemmed from using the util/gem5img.py script to create a new disk image. The second set of issues stemmed from the fact that Ubuntu Core no longer exists -- the thousands of links to Ubuntu Core that currently exist on the internet will 404. I eventually discovered that Ubuntu Core is now Ubuntu Base. If your current O.S. is Ubuntu 14.04, you can follow Jason's tutorial without much trouble if you use ubuntu-base instead of ubuntu-core.

If you have Ubuntu 16.04 or some other version that has updated sfdisk then you won't be able to use util/gem5img.py. The reason is that several options for sfdisk, such as -D, -C-, -H, and -S have been removed. The script uses sfdisk to create a partition on the newly created disk image. So I had to go about creating my own disk image the old fashioned way. The old fashioned way involves using dd to create an image of all 0s then using GParted GUI to create a partition. GParted allows you to create an MBR partition (select msdos for the partition table type) and will allow you to choose the ext2 filesystem (which util/gem5img.py defaults to).

Once I had created my disk image and partitioned it, I needed to mount it before I could continue with Jason's tutorial. For this we need to know the offset at which the partition starts, which we can do using parted. For an image called ubuntu-14.04.img, we (1) launch parted, (2) specify the units to be Bytes, and (3) print information on the partition:

> parted ubuntu-14.04.img
WARNING: You are not superuser.  Watch out for permissions.
GNU Parted 3.2
Using /home/badrmari/Projects/uoft-comp-arch/gem5/disks/ubuntu-14.04.img
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) unit
Unit?  [compact]? B
(parted) print
Model:  (file)
Disk /home/badrmari/Projects/uoft-comp-arch/gem5/disks/ubuntu-14.04.img: 8589934592B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start     End          Size         Type     File system  Flags
 1      1048576B  8589934591B  8588886016B  primary  ext2

We can see from above that the Start is at 1048576 Bytes. So now we need to mount the device at that specific offset:

> mount -o loop,rw,offset=1048576 ubuntu-14.04.img mnt/

You may now proceed with Jason's tutorial and extract Ubuntu Core into mnt, et cetera. I also removed the plymouth-upstart-bridge.conf from the image's /etc/init to avoid some error/warning messages. Placing the image into your $M5_PATH/disks directory, you can then load it in gem5 (with the kernel created in the previous post):

> ./build/X86_MESI_Two_Level/gem5.opt configs/example/fs.py --kernel=x86_64-vmlinux-3.4.112 --disk-image=ubuntu-14.04.img

Updating the gem5 Linux Kernel for x86

I was recently pointed to a GitHub repository for setting up a disk image for gem5 that contains the PARSEC Benchmarks. Unfortunately the steps listed to create a disk image did not work for me when booting with gem5. However, the repo does contain a kernel configuration for linux version 3.4.112, which I have saved in a gist just in case.

To compile the kernel, head over to The Linux Kernel Archives and download the appropriate version (in this case 3.4.112). Extract the kernel, and copy over the kernel configuration into the extracted directory with the .config filename. Now typing make vmlinux should give you a kernel that you can boot just fine with gem5, even while using their (very old) disk images.

The following steps should produce a binary called vmlinux:

> wget https://cdn.kernel.org/pub/linux/kernel/v3.x/linux-3.4.112.tar.xz
> tar -xJf linux-3.4.112.tar.xz
> cd linux-3.4.112
> wget https://gist.githubusercontent.com/mariobadr/8f12d7fd5794e6b219d8cc9c193c0874/raw/3e529e91b341597697ec1364a0a7766296e54632/X86_64-vmlinux-3.4.112
> mv X86_64-vmlinux-3.4.112 .config
> make vmlinux

In order to load the kernel add it to your binaries directory in M5_PATH -- I renamed it to x86_64-vmlinux-3.4.112. Because of the way the full system python scripts work, you'll need to add the full system files from the gem5 website to their respective binaries and disks folders. That is, $M5_PATH/binaries should contain x86_64-vm-linux-2.6.22.9 and in addition to the kernel we created above. Also, M5_PATH/disks should contain x86root.img and linux-bigswap2.img (the latter is actually in the ALPHA full system tarball) disk images. Ideally the python scripts should be updated... but that's a different post entirely. Assuming your directories are setup as above and you have compiled an X86 version of gem5, you can run gem5 in full system mode like so:

> ./build/X86_MESI_Two_Level/gem5.opt configs/example/fs.py --kernel=x86_64-vmlinux-3.4.112