Unikraft Porting Libraries Walkthrough
In order to port an existing application on top of Unikraft, one has to port its dependencies using Unikraft external libraries. Each external library will have its own project or repository and, ideally, no source file will be necessary. Of course, there are exceptions which we will talk about later in this article. The most difficult part when creating an external library is translating the original build system of the library to Unikraft build system. In other words, one has to understand how the original build systems works in order to identify which source files are compiled, how the generated file objects are linked together and which flags are used for these operations. If the original build system is Makefile based, then this task is pretty straight-forward. However, with more complex build systems this task becomes more difficult, especially if the person doing the porting is not a developer of the original code.
This article presents the steps involved in creating an external library. We will be using the
pcre library as an example throughout this article. We chose
pcre because it is a popular dependency among Debian/Ubuntu packages, it has a small number of source files and it required only 3 commits to fully port it for Unikraft.
- 1 Steps for creating the external library
- 2 Further information
Steps for creating the external library
Every external library should have the following files:
. ├── .gitignore ├── CODING_STYLE.md ├── CONTRIBUTING.md ├── COPYING.md ├── Config.uk ├── MAINTAINERS.md ├── Makefile.uk ├── README.md └── exportsyms.uk
Some of the files will be exactly the same for all external libraries, some will have some small changes and some will be completely different. For simplicity, we propose the commits we created for
pcre to be adapted for your libraries as well. We will describe each commit in the following subsections.
Initial empty commit (optional)
It is always a good idea to start with an empty commit because it helps on rebasing when amending commits.
Introducing the library skeleton
You can simply copy and adapt the first commit of
pcre, which introduces the library skeleton. The
exportsyms.uk (we do not export any symbol for now) will be exactly the same. All you have to do is:
- adapt the symbol names in
- change the maintainers in
- pick a license to be used throughout the external library and copy it in
COPYING.md; for example,
pcreuses BSD-3-clause license
The second commit should add the Unikraft
Makefile.uk file. This is the most important file, similar to regular Makefiles, it contains the list of source files to be compiled and the building flags. The
Makefile.uk file of
pcre external library has the following sections:
- Library registration - This is where you register your new library to the Unikraft build system. We have two registration directives there, the second one is commented out because it would register the glue code as a library of its own. In exceptional case, one would need to add some glue code in order to make the ported library work with Unikraft (we will revisit this below). Fortunately, for
pcrethis wasn’t the case.
- Original sources - Here are the variables which keep the information about the location of the archive containing the original code. The fetching directive does just that: fetching the compressed archive and decompressing it. You will notice that there is also a patching directive which is not doing anything for
pcre. There are other exceptional cases when patches must be applied to the original code in order to make it work with Unikraft. Usually, these patches correct some wrong doings in the origin code: bugs, code that doesn’t comply with standards, code that is enabled for well-known operating systems and should also be enabled for Unikraft, etc. You can find more information about patches below.
- Helpers - Here is where you define auxiliary variables, if you need any.
- Library includes - These are the include paths that should be captured from the original build. The external library might need its own headers directory (here represented by
$(LIBPCRE_BASE)/include), but this is not the case for
- Global flags - You will find that quite a lot of libraries have a lot of warnings. Patching the code wouldn’t be the smartest idea in that case, so that’s why we should choose to disable warnings that don’t indicate critical issues. Here is also where you should add the compilation flags and the preprocessing symbol definitions (e.g.
pcre) which are required for the ported library.
- Glue between Unikraft and pcre - As we mentioned it above, there might be some cases where some glue code is required to make things work. This is where you write which source files contain the glue code.
- Library code - This contains the list of original source files. Be careful to select only those source files which are required for the library you want to build. An original build may generate multiple libraries and executables. For example,
pcreoriginal code contains two libraries and two test executables, so we had to pick only the source files of the main library.
- Library prepare - Some libraries might require additional steps before compiling. For example,
autotoolsbased libraries need
./configureto be run before running
make. Some source files might be generated, other might be just symbol links to existing files. This section contains all the custom rules required for building the original code. Notice that the first custom rule should depend on the
$(LIBNAME_BUILD)/.originfile which will be created after decompressing the archive. In the
pcreMakefile we used
LIBPCRE_PREPARED_DEPSjust as an auxiliary variable, this is not mandatory. The last two directives are always mandatory for prepare rules in order to be properly registered in the Unikraft build system.
Add patches for the original code (optional)
As mentioned in the previous section, sometimes the original code has to changed in order to make it compile or to fix some bugs. What you have to do is to create patches which will be automatically applied by the Unikraft build system after downloading the source code. All the patches should be put in the
patches/ subdirectory, named in order. You can find plenty of examples in our previous projects, e.g. newlib, pthread-embedded. In order to create the patches, one way is to create a local git repository for the original code after the Unikraft build systems downloads it and commit the changes there.
exportsyms.uk file for exporting library symbols, such as function and variable names. The list of exported symbols should contain only the API symbols, meaning that one has to get familiar with the library in order to know the API. The library itself might have more global symbols, but the extra ones will have the visibility updated to private by the Unikraft build system. For
pcre we used the symbols listed in its API documentation.
Please keep the symbols sorted alphabetically so that the reviewing may be done easier.
Adding extra configuration symbols (optional)
An optional step after making the changes above would be to add some configuration symbols which would be used in enabling and configuring features of the ported library. You may find a good example in the
Config.uk file of our
pthread-embedded port which adds configuration symbols for two features in the original code and for building its unit tests.
Adding extra information (optional)
For some external libraries, one would need additional information about how to use it. For example,
pthread-embedded library needs to stay before
newlib when adding them to the value of
LIBS variable in
Makefile file of the application. Information like this would have to stay in the
README.md, therefore a new commit adding this information would be needed.
- A walkthrough of the build system can be found here. The slide set explains how Unikraft's build system can be fed to do the job.
- You can find more information about the syntax and the predefined variable names of
Makefile.ukfiles on the Unikraft developer's guide or in the guides under the
doc/subdirectory in the source tree.