2012.07.31 - custom C/C++ apps for OpenWRT

some time ago i wrote about nice TP-Link's WR703 router, that happened to be feature rich and low priced. currently i'm playing around with it, while creating a new project (details to be released soon). OpenWRT does support this device, so new firmware installation was fast. the only problem was that the router's original firmware had only Chinese language support – you need to know where to click. fortunately page names are in english, so by highlighting the link you can find proper menu.

after installation one may now want to create own software. this is a bit tricky. there is ready-to-go openwrt's image, but there are “but(t)s”… there are also different solutions available, for MIPS. since it took my about a day before it all started to work i'll present the working solution, with the example, so that you can start faster.

GCC supports MIPS. this is good. first thing i've tried was EmDebian. nice, cool, easy to start… but this is a solution for devices with bigger drives – minimal system image is about 80-90MB. since this device has 4MB flash, you can skip this toolchain this time…

so we're back to OpenWRT's solutions. there are available, already compiled packages for linux: SDK, ImageBuilder, Toolchain. though they are heavy (1G total), they are neat… if you happen to have 32-bit x86 machine (alt. VM). if you happen to have amd64 (as most ppl nowadays do, including myself) you're out of luck and need to compile stuff by our own.

in theory all of this is described on the net, but it's kind of tricky, since most of the times documents are not up to date. some of them are plain wrong or nasty hacks that will work for owner's machine, with one version of toolchain only, etc. what i wanted is to present you with the working solution. so lets start…

first step is to download current version from OpenWRT's SVN repository:

svn co svn://svn.openwrt.org/openwrt/trunk openwrt-trunk
cd openwrt-trunk

now you need to download/update packages and prepare them for the installation:

./scripts/feeds update -a
./scripts/feeds install -a

this will take a while and will download some data from the net – be patient. having this done it is a time for the configuration of what you want to build. if you have compiled linux's kernel before, you'll find this step very convenient:

make menuconfig

the most important thing here is to select your platform (“Target System” option) – TP-Link WR703 is “Atheros AR7xxx/AR9xxx”. since we'll be building for C and C++ it is good to choose uClibc++ as a default standard library for C++ as well – “Global build settings” → “Preferred standard C++ library” → “uClibc++”. now you can build whatever is needed for our minimal installation:

make -j `grep processor /proc/cpuinfo | wc -l`

this step will take about 1h (yes – an hour). most of the stuff is compiled sequentially, so multicore arch will not help you much. you need to wait…

when the main part is done, it is the time for preparing own package.

all packages are placed under package directory. it is enough to copy proper directory structure there. “proper structure” is the most tricky part – i've spent most of the above mentioned time on it. to make your life simpler i've created a working helloworld example for you to test. unpack it into the package directory (i.e. create new package, named helloworld). the content of this directory follows:

1. package/helloworld/Makefile – makefile, that openwrt's toolchain recognizes.
2. package/helloworld/src/Makefile – your own makefile; it is enough if it builds as a default target, and has clean target.
3. package/helloworld/src/helloworldpp.cpp – C++ hello-world program.
4. package/helloworld/src/helloworld.c – C hello-world program.

C and C++ sources are regular files. Makefile, inside the src directory, can be of your choice (i.e. any), as long as it is able to build your code. the nice thing about OpenWRT's framework is you can do:

cd package/<mypackage>/src
make

and this will compile code for your local machine (as usually Makefiles do), while building with OpenWRT's toolchain will cross-compile it.

the real magic is hidden inside the Makefile, located in the same directory as the src directory. here is an example file, taken from the helloword example package:

include $(TOPDIR)/rules.mk

PKG_NAME:=helloworld
PKG_RELEASE:=1

include $(INCLUDE_DIR)/uclibc++.mk
include $(INCLUDE_DIR)/package.mk

define Package/$(PKG_NAME)
  SECTION:=utils
  CATEGORY:=Utilities
  TITLE:=helloworld printing program in C and C++
  DEPENDS:=+uclibcxx
endef

define Build/Prepare
      mkdir -p $(PKG_BUILD_DIR)
      $(CP) ./src/* $(PKG_BUILD_DIR)/
endef

define Build/Configure
endef

define Build/Compile
      $(MAKE) -C $(PKG_BUILD_DIR) $(TARGET_CONFIGURE_OPTS)
endef

define Package/$(PKG_NAME)/install
      $(INSTALL_DIR) $(1)/bin
      $(INSTALL_BIN) $(PKG_BUILD_DIR)/helloworld   $(1)/bin/
      $(INSTALL_BIN) $(PKG_BUILD_DIR)/helloworldpp $(1)/bin/
endef

$(eval $(call BuildPackage,helloworld))

first line includes required elements from the OpenWRT's build. PKG_* are your local settings (here: name and version, but there are more available).

next two includes are very important. first includes uclibc++.mk, that switches to uClibc++ as the default, standard library, for your C++ programs (note that uClibc++ is smaller than regular libstdc++). next include (package.mk) is required to build packages. note that the order of these includes DOES matter.

next step is define Package/$(PKG_NAME) section. you can put your package into the menu (remember make menuconfig?) and, more importantly, say what are your dependencies (required step – otherwise you'll get an errors from the build). here:

DEPENDS:=+uclibcxx

say's we depend of uClibc++ (required by C++ version of helloworld program – helloworldpp).

define Build/Prepare section defines how to prepare your package for building. copying sources to the destination (i.e. build) dir is fine for us.

define Build/Configure can be used to run ./configure script, if required.

define Build/Compile is an important one – it tells build how to start build system of your package. it is important how this is done – $(TARGET_CONFIGURE_OPTS) is mandatory. if your makefile builds by default, presented section's content is just fine.

define Package/$(PKG_NAME)/install tells how to install the package, that has been build. here we simply copy (using install command) both compiled binaries to the output directory.

last line of the makefile ($(eval $(call BuildPackage,helloworld))) does all the magic, via specially prepared macro. just keep it the way it is. :)

also keep in mind that make does differ between tabulators and spaces! content of the following sections must be tab-indented:

• define Build/Prepare
• define Build/Configure
• define Build/Compile
• define Package/$(PKG_NAME)/install

after having your package added you must enable it in the menu. the simplest way to do this is:

make oldconfig

this will update your previous configuration and prompt you what to do with new element – that is: helloworld. enable it and your ready. you can now build it.

in order to build, enter main directory (i.e. openwrt-trunk) and run:

make package/helloworld/compile

to clean just run:

make package/helloworld/clean

by default you'll not see any details, printed by your makefile. in case something goes wrong, just add V=99 to the command line:

make package/helloworld/compile V=99

in order to run binary on your OpenWRT machine you need to copy them there:

scp helloworld helloworldpp root@router:/tmp/

then ssh to the machine and install uClibc++ (if it is not there yet):

opkg update
opkg install uclibcxx

now you can run your code:

ssh root@router
cd /tmp
./helloworld
./helloworldpp

that's it – have fun! :)

out of dozens of pages i've read, i've choose some that are the most interesting, for building with OpenWRT's build system:

1. http://hype-free.blogspot.com/2009/04/compiling-software-for-openwrt-and.html – near-to-ready example package.
2. http://wiki.openwrt.org/doc/howto/obtain.firmware.sdk – info on SDK usage, for those who have x86 machines. ;)
3. http://wiki.openwrt.org/doc/devel/packages – official how to on creating packages; many options described.
4. http://wiki.openwrt.org/doc/howto/buildroot.exigence – information on how to build buildroot.