===============================
                    Running Numptyphysics on Genode
                    ===============================


                             Norman Feske


Numptyphysics is a simple yet wonderful game where the player has to use
gravity to solve puzzles. I figured that it would be nice starting point for
my 8-years old son to explore the Raspberry Pi. Hereby, I'm documenting the
steps that were needed to run this game on Genode on this platform. See the
game's website for more information:

:[http://numptyphysics.garage.maemo.org/]:

  Website of Numpty Physics


Getting acquainted with the code and integrating it with Genode
===============================================================

The first step of porting the game is creating a so-called port file for
downloading the source code. Let us create a new port file at
_world.git/ports/numptyphysics.port_ with the following content:

! LICENSE   := GPLv3
! VERSION   := git
! DOWNLOADS := numptyphysics.git
!
! URL(numptyphysics) := https://github.com/thp/numptyphysics.git
! DIR(numptyphysics) := src/app/numptyphysics
! REV(numptyphysics) := 336729c44c9ced3907d049cc1c1130a26c5708a3

The license information is taken from the COPYING file in the top-level
directory of the git repository. The REV(git) is hash of the current version.
In addition to the ports file, we need a hash file _ports/numptyphysics.hash_
with the following (temporary) content.

! dummy

With both the port file and the hash in place, we can test the downloading
of the port using Genode's 'prepare_port' tool:

! <genode-dir>/tool/prepare_port numptyphysics CHECK_HASH=no

The downloaded source code can be found at
_<genode-dir>/contrib/numptyphysics-dummy_.
Before continuing the actual Genode port, let us first build and run the
game on Linux to learn more about the build procedure and eventual
dependencies. The sourc tree at _src/app/numptyphysics/_ comes with a plain
makefile. Let's give it a try.

! make 2>&1 | cat > build.log

We redirect the build output to the _build.log_ file to be able to examine
it later.
According to the build commands, the game uses libSDL, which is nice because
libSDL already exists on Genode. At the link stage, we futher see that the
libraries SDL_ttf, SDL_image, and zlib are used.

The build process is fairly simple, all files of the top-level directory and
the _Box2D/_ directory are incorporated. Additionally, the
_os/OsFreeDesktop.cpp_ file is used, which presumably contains the OS
abstraction.


Compiling the program using Genode's build system
=================================================

With this information, we can start to create a
_world.git/src/app/numptyphysics/target.mk_ file as follows:


! TARGET := numptyphysics
!
! NUMPTY_DIR := $(call select_from_ports,numptyphysics)/src/app/numptyphysics
!
! SRC_CC := $(notdir $(wildcard $(NUMPTY_DIR)/*.cpp))
!
! vpath %.cpp $(NUMPTY_DIR)

The first line declares the name of the target. The following line queries the
location of the downloaded source code and stores the path in the variable
'NUMPTY_DIR'. The following line collect a list of all files with the 'cpp'
extension in the 'SRC_CC' variable. The last line tells the build system where
to look for the source codes.

We further need to add the source codes of the _Box2D_ directory. The list
of needed cpp files can be obtained by the examining the build.log file that
we produced when building the game on the host.

! SRC_CC += Box2D/Source/Dynamics/b2Body.cpp \
!           Box2D/Source/Dynamics/b2Island.cpp \
!           ...

To try out the _target.mk_ file, let us create a build directory for the
Linux base platform using the 'create_builddir' tool and add the our "world"
repository to the build configuration by adding the following line to the
_etc/build.conf_ file:

! REPOSITORIES += $(GENODE_DIR)/repos/world.git

Now, let's try to build the target

! make app/numptyphysics

We get a build error like this:

! fatal error: Box2D.h: No such file or directory

This error occurs because we have not declared any include directories.
Let us add the required declarations to the _target.mk_ file:

! INC_DIR += $(NUMPTY_DIR) $(NUMPTY_DIR)/Box2D/include

Now, we get another error:

! fatal error: assert.h: No such file or directory

The _assert.h_ header comes from the libc, which we haven't incorporated
yet. So let's add the following line:

! LIBS += libc

The next build attempt stops with:

! fatal error: cmath: No such file or directory

The _cmath_ header is provided by the standard C++ library. Let's add it
to the LIBS declaration.

! LIBS += stdcxx

Now the build proceeds nicely until we hit the following problem:

! fatal error: SDL/SDL.h: No such file or directory

We know by now how to resolve this error, don't we?

! LIBS += sdl

Expecting that we will need to repeat the same procedure for
SDL_image, SDL_ttf, and zlib. So we might add those libraries right away.

The next error, however, is less easy to overcome:

! fatal error: X11/X.h: No such file or directory

We don't want to run the program under X11 but on Genode. After examining the
file _Canvas.cpp_ (the one where the compilation error occurred), we can see
that the X11 related code is just some quirk that is optional. I.e., there is
no counterpart for the WIN32 version. To fix this issue, we have two options.
Either, we change the original code or we emulate the X11 APIs that are
expected by the code. That is, we could provide custom headers named "X11/X.h"
and "X11/Xlib.h" that contain dummy definitions of the API calls and types
used by _Canvas.cc_. In this case, I opted for a small patch of the original
source code:

! diff --git a/Canvas.cpp b/Canvas.cpp
! index c0bddf1..78b1f20 100644
! --- a/Canvas.cpp
! +++ b/Canvas.cpp
! @@ -28,9 +28,11 @@
!  #include "Swipe.h"
!  #include <SDL/SDL_syswm.h>
!  #ifndef WIN32
! +#ifndef GENODE
!  #include <X11/X.h>
!  #include <X11/Xlib.h>
!  #endif
! +#endif
!  #undef Window
!  
!  //#define FORCE_16BPP
! @@ -759,7 +761,7 @@ void Window::raise()
!    SDL_VERSION( &sys.version );
!    SDL_GetWMInfo( &sys );
!  
! -#if !defined(WIN32) && !(defined(__APPLE__) && defined(__MACH__))
! +#if !defined(WIN32) && !defined(GENODE) && !(defined(__APPLE__) && defined(__MACH__))
!    /* No X11 stuff on Windows and Mac OS X */
!  
!    // take focus...

To make this patch work, we need to add the definition of GENODE to our
_target.mk_ file:

! CC_OPT_Canvas += -DGENODE

By appending the suffix '_Canvas' to the 'CC_OPT' variable, we can customize
the compile flags for the individual compilation unit. With this change, the
compilation proceeds nicely until compiling _Swipe.cpp_.

! Swipe.cpp:6:22: fatal error: X11/Xlib.h: No such file or directory

The symptom looks similar to the one of _Canvas.cpp_ so we likely need to
patch the source. However, when looking at the file, we see that everything
contained therein is actually related to X11. So let us simply skip the
file by tweaking our 'SRC_CC' definition:

! SRC_CC := $(filter-out Swipe.cpp,$(notdir $(wildcard $(NUMPTY_DIR)/*.cpp)))

Of course, the omission of this file will likely result in a linker error
because other parts of the program will refer to functions or variables
defined in it. But let us save this problem for later. However, we hit another
problem on our next attempt to build:

! fatal error: help_text_html.h: No such file or directory

When revisiting the _build.log_ of the original build process for Linux,
we can see that this file is generated from _help_text.html_ using 'xxd'.
We can do the same in our _target.mk_ file:

! Dialogs.o: help_text_html.h
! help_text_html.h: help_text.html
! 	$(VERBOSE)(cd $(NUMPTY_DIR); xxd -i help_text.html) > $@

Now, the reach the linking stage:

! warning: cannot find entry symbol main; defaulting to 0000000001000000

This is where the "os" backend comes into play. Let us add the
_os/OsFreeDesktop.cpp_ file to the build.

! SRC_CC += os/OsFreeDesktop.cpp

Now, we get linker errors as we expected when having omitted the
_Swipe.cpp_ file:

! Canvas.cpp:663: undefined reference to `Swipe::m_syswminfo'
! Canvas.cpp:666: undefined reference to `Swipe::lock(bool)'
! Dialogs.cpp:271: undefined reference to `Swipe::lock(bool)'
! Dialogs.cpp:271: undefined reference to `Swipe::lock(bool)'
! Dialogs.cpp:265: undefined reference to `Swipe::lock(bool)'

To resolve these symbols, we create a new file _app/numptyphysics/dummy.cc_:

! #include "Swipe.h"
! 
! SDL_SysWMinfo Swipe::m_syswminfo = { 0 };
! 
! void Swipe::lock(bool locked) { }

We also need add this file to our 'SRC_CC' declaration

! SRC_CC += dummy.cc
! vpath dummy.cc $(PRG_DIR)

After this step, the program links successfully!


Testing the program
===================

To run the game, we need to create a run script. The easiest way is to take
template and modify it according to our needs. The _libports/run/sdl.run_
script is a good starting point. So let's copy this file to
_world.git/run/numptyphysics.run_ and give it a try from within the
build directory:

! make run/numptyphysics

We need to customize the run script in the following ways:

* Adding _app/numptyphysics_ to the 'build_components', and removing
  _test/sdl_
* Replacing the "test-sdl" entry of the init config with an entry
  for "numptyphysics".
* Adding "numptyphysics" and the needed shared libraries to the
  boot modules to be included in the boot image. The shared libraries
  are
  ! freetype.lib.so
  ! jpeg.lib.so
  ! ld.lib.so
  ! libc.lib.so
  ! libm.lib.so
  ! libpng.lib.so
  ! pthread.lib.so
  ! sdl_image.lib.so
  ! sdl.lib.so
  ! sdl_ttf.lib.so
  ! stdcxx.lib.so
  ! zlib.lib.so

When executing the run script, we see a blank screen. Issuing 'dmesg'
reveals that the program just crashed because of a null pointer:

! segfault at 0 ip 011b294e sp 400ff840 error 4 in sdl_ttf.lib.so[11b0000+5000]

To debug the problem, let us attach GDB to the process. We can do that by
adding the following code at the beginning of the 'main' function in
_os/OsFreeDesktop.cpp_:

! extern "C" void wait_for_continue();
!
! int main(int argc, char** argv)
! {
!   wait_for_continue();
!   ...

On Linux, this code will pause the program until the user presses the
enter key. When executing the run script again, the scenario will stop
right at the startup of numptyphysics. This gives us the opportunity
to attach GDB to the program:

! gdb -p $(pidof "[Genode] init -> numptyphysics") \
!     <build-dir>/bin/numptyphysics

In GCB, we issue the "c" (continue) command. Now, when pressing enter
in the window where we started the Genode scenario, GDB will respond
with a message like:

! Program received signal SIGSEGV, Segmentation fault.

Unfortunately, the attempt to show the backtrace via the "bt" command
results in just:

! (gdb) bt
! #0  0x011b294e in ?? ()

Not very helpful. The reason GDB cannot obtain a meaningful backtrace
is that the program was compiled while omitting frame pointers, which is
the default. So let's disable the omission of frame pointers by
adding the following line to _<build-dir>/etc/tools.conf_. Because the
file does not exist yet, you have to create it.

! CC_OLEVEL := -fno-omit-frame-pointer

When repeating the steps above, we can obtain a full backtrace.
It turns out that the 'Font::titleFont()' function tried to load a font
called "femkeklaver.ttf", which we does not exist in our Genode world.
We have to add this font to the virtual file system as seen by program.
There is likely more data missing, i.e., all the files contained in
the _/numptyphysics/data/_ directory. To provide those files to the program,
we have to take two steps. First, we have to bundle them in a TAR archive,
and second, we will have to "mount" this TAR archive into the program's
virtual file system. Creating a TAR archive can be done as a side effect
of compiling the program by adding the following lines to the _target.mk_
file:

! $(TARGET): numptyphysics_data.tar
! numptyphysics_data.tar:
! 	$(VERBOSE)cd $(NUMPTY_DIR)/data; tar cf $(PWD)/bin/$@ .

When issuing 'make app/numptyphysics' the next time, the TAR archive will
appear in the _bin/_ directory. To "mount" the TAR archive's content into
the program's VFS. we need to modify the _run/numptyphysics.run_ script.

* Add 'numptyphysics_data.tar' to the list of 'boot_modules'
* Add a VFS configuration for the "numptyphysics" process:
  ! <libc stdout="/dev/log" stderr="/dev/log" >
  !   <vfs>
  !     <tar name="numptyphysics_data.tar" />
  !     <dir name="dev"> <log/> </dir>
  !   </vfs>
  ! </libc>
  Note that we also enabled the redirection of stderr to Genode's LOG service.

When executing the run script again (don't forget to press enter because the
'wait_for_continue' is still in place), we see that the segmentation fault
is gone but we are presented with another error:

! [init -> numptyphysics] C++ runtime: std::out_of_range
! [init -> numptyphysics] C++ runtime: basic_string::substr

It would be useful if we stopped the execution on the occurrence of the
'basic_string::substr' exception. We can do that be tweaking Genode's
C++ runtime a bit at the code that prints the error message. The code
is located in the function 'fputs' in _repos/base/src/base/cxx/misc.cc_.
Just add the following two lines to hold the program when the second
message appears:

! if (strcmp("basic_string::substr", s) == 0) {
!   Genode::error("stop!");
!   for (;;);
! }

When running the scenario again, we will see the "stop!" message. When
attaching GDB to the process and printing the backtrace, we can see where the
exception originated. It turns out that the name returned by the
'levelName' function is "err". The subsequent attempt to strip the suffix
".npd" from the string using 'substr' with the string length - 4 triggers
the out-of-range exception. We have to answer the question of why the
'levelName' function returns this strange string. By following the code
paths and adding minor instrumentations, we learn that the 'findLevel'
function returns a null pointer for the level 0 because the
'm_collections' array has a size of 0. So the initialization of this
array has likely gone wrong.
The problem originates from ill-defined configuration values in 'Config.h'.
Adding the following line to the _target.mk_ file solves it:

! CC_OPT += -DINSTALL_BASE_PATH='"/"' -DUSER_BASE_PATH='"/"'

On the next attempt to execute the run script, we will see the title
screen of the game!

Next, let us try it on a microkernel by creating a build directory,
tweaking the _etc/build.conf_ file (e.g., adding the _world.git_
repository), and executing the run script.

Strangely, we encounter an error quite early when the program starts:

! C++ runtime: std::logic_error

Instrumenting the initialization code of _App.c_ pinpoints the problem to
a call of 'getenv("HOME"). Because on Genode, no environment variables are
set, the function returned 0, which could not be handled by the surrounding
C++ string manipulation code. To work around this problem, we override the
default implementation of the libc's 'getenv' function with a dummy placed
in a new _getenv.cc_ file that we link to the target:

! extern "C" char *getenv(const char *name)
! {
!   Genode::log("environment variable \"", name, "\" requested");
!   return (char *)"";
! };

Now that we have a first somehow usable state of the game, let us preserve
the little X11-related patch. Because we cloned the 3rd-party code from
GitHub, obtaining a patch with our changes is easy:

! cd <genode-dir>/numptyphysics-dummy/src/app/numptyphysics
! git diff > <genode-dir>/repos/world.git/src/app/numptyphysics/canvas.patch

To apply the patch automatically when installing the port, we adjust the
header of the patch from

! diff --git a/Canvas.cpp b/Canvas.cpp
! index c0bddf1..78b1f20 100644
! --- a/Canvas.cpp
! +++ b/Canvas.cpp

to

! +++ src/app/numptyphysics/Canvas.cpp

Futhermore, we specify the patch in _world.git/ports/numptyphysics.port_
as follows:

! PATCHES := src/app/numptyphysics/canvas.patch

The next time, the port is installed via the 'prepare_port' tool, the patch
will be applied.

Alternatively to maintaining a patch, in particular if we required more
profound changes, we could consider maintaining a fork of the numptyphysics
Git repository and fetch the source code from there.

The last step of the port is generating the correct hash sum of the port
and committing the port to the _world.git_ repository.

! <genode-dir>/tool/ports/update_hash numptyphysics

This command will compute the hash value of the current version of the
port and update the file _world.git_/ports/numptyphysics.hash_ accordingly.
Hence, the next time, you execute the 'prepare_port' command, the
'CHECK_HASH=no' argument can be omitted.