[RFC] add oonf_api to pkg #630

benpicco · 2014-02-06T00:53:39Z

This adds OLSRs oonf_api as a RIOT package. oonf_api provides a rfc5444 reader/writer as well as utilities such as an avl tree, a flexible buffer, a double linked list as well as address abstraction (IPv4/Ipv6/MAC - not very necessary on RIOT) and string functions.
The package provides three modules, oonf_common, oonf_rfc5444 (depends on oonf_common) and cunit for unit tests.
This package pulls in the whole of oonf_api from git, but the majority of the code (OS abstraction, config framework, telnet and webserver, …) is neither required nor functional on RIOT, so it's ignored by the build system.
The only folders of interest are:

dir	use
src-api/common	library functions
src-api/rfc5444	rfc5444 reader/writer
tests/cunit	cunit test framework
tests/common	unit tests for common (run `generate_makefiles.sh`)
tests/rfc5444	unit tests for rfc5444 (run `generate_makefiles.sh`)
examples/rfc5444_reader_writer	example application

Usage is a bit cumbersome because of the include paths, as seen in https://github.com/benpicco/tiny_olsr2/blob/master/Makefile

EXTERNAL_MODULES +=$(RIOTBASE)/pkg/oonf_api
USEMODULE += oonf_common
USEMODULE += oonf_rfc5444
INCLUDES += -I$(RIOTBASE)/pkg/oonf_api/oonf_api/src-api

Also, build order is important, modules that depend on this won't find includes unless the pkg is build, aka checked out and patched.

I'm welcome for suggestions on how to improve this.

mehlis · 2014-02-06T21:20:13Z

@Lotterleben is this working for you? please check carefully for your use case.

LudwigKnuepfer · 2014-02-06T22:42:36Z

I think it makes sense to wait for the outcome of #576 before merging this.

Lotterleben · 2014-02-09T14:45:47Z

@mehlis seems to be working fine

mehlis · 2014-02-09T18:12:50Z

👍 to wait until #576 is discussed and merged

benpicco · 2014-02-12T19:02:06Z

rebased against #576

mehlis · 2014-02-12T21:13:20Z

perhaps this is unrelated but: would it make sense to have a rfc5444 "service" running in its own thread. seems to me that olsr/aodv and nhdp need this parser.

Is this library thread safe? What is its footprint in terms of stack usage? Does it use malloc/free?

OlegHahm · 2014-02-15T17:23:05Z

I wasn't able to build this. Can you provide an example application? master of https://github.com/benpicco/tiny_olsr2 seems to be not up-to-date.

benpicco · 2014-02-18T16:40:00Z

I will move tiny_olsr2 to sys/net/routing/olsr2 and add an example application to projects, but for now you can run make patch in pkg/oonf_api and then build oonf_api/examples/rfc5444_reader_writer/ or oonf_api/tests/common/ and oonf_api/tests/rfc5444/ if you just want to try out oonf_api.

benpicco · 2014-02-18T16:57:18Z

The rfc5444 reader/writer is based on callback functions for each action, so it might be possible to switch this over to message passing, but then again the library is not thread safe (however the portions used don't keep any state on their own) so additional efforts would be required. And I don't think the overhead of having another thread would be justified, in the most common case you won't run olsr2 and aodv2 in parallel.

size for the ARM build says

   text    data     bss     dec     hex filename
   1364       0       0    1364     554 autobuf.o (ex msba2/oonf_api.a)
   2540       0       0    2540     9ec avl.o (ex msba2/oonf_api.a)
    120       0       0     120      78 avl_comp.o (ex msba2/oonf_api.a)
      0       0       0       0       0 daemonize.o (ex msba2/oonf_api.a)
   4080       0       0    4080     ff0 netaddr.o (ex msba2/oonf_api.a)
   1136       0       0    1136     470 netaddr_acl.o (ex msba2/oonf_api.a)
    676       0       0     676     2a4 rfc5444.o (ex msba2/oonf_api.a)
    796       0       0     796     31c rfc5444_context.o (ex msba2/oonf_api.a)
     40       0       0      40      28 rfc5444_iana.o (ex msba2/oonf_api.a)
   8056       0       0    8056    1f78 rfc5444_msg_generator.o (ex msba2/oonf_api.a)
   1584       0       0    1584     630 rfc5444_pkt_generator.o (ex msba2/oonf_api.a)
   3288       0       0    3288     cd8 rfc5444_print.o (ex msba2/oonf_api.a)
   6080       0       0    6080    17c0 rfc5444_reader.o (ex msba2/oonf_api.a)
    512       0       0     512     200 rfc5444_tlv_writer.o (ex msba2/oonf_api.a)
   4152       0       0    4152    1038 rfc5444_writer.o (ex msba2/oonf_api.a)
   3728       0       0    3728     e90 string.o (ex msba2/oonf_api.a)
   1260       0       0    1260     4ec template.o (ex msba2/oonf_api.a)
  39412       0       0   39412    99f4 (TOTALS)

but calloc is used. After startup about 4k of autobuf buffers are allocated, they grow dynamically with realloc if more memory is needed. Their initial size is getpagesize() (arbitrarily set to 512 as it's not available on the msba2), but smaller values might be used.

Lotterleben · 2014-02-19T15:38:49Z

The Makefile in RIOT/pkg/oonf_api/oonf_api/examples/rfc5444_reader_writer/ uses a path to the RIOTBASE that probably won't work for everybody:

export RIOTBASE =/home/benpicco/µkleos/riot/RIOT

Lotterleben · 2014-02-19T15:46:54Z

After correcting the RIOTBASE to export RIOTBASE =../../../../../ I got the following error:

 gcc: error: /home/lotte/bachelor/riot/RIOT/pkg/oonf_api/oonf_api/examples/rfc5444_reader_writer/bin/native/oonf_common.a: No such file or directory
 gcc: error:  /home/lotte/bachelor/riot/RIOT/pkg/oonf_api/oonf_api/examples/rfc5444_reader_writer/bin/native/oonf_rfc5444.a: No such file or directory

benpicco · 2014-02-19T17:22:32Z

oops, missed that one - should work now.

Lotterleben · 2014-02-24T14:44:51Z

When I tried to compile des-testbed/RIOT-projects#98, I ran into the following errors:

make complained that it couldn't find the oonf api:

lotte@riot:~/bachelor/riot/projects/olsr2$ make
"make" -C /home/lotte/bachelor/riot/RIOT/pkg/oonf_api
make[1]: Entering directory `/home/lotte/bachelor/riot/RIOT/pkg/oonf_api'
make[1]: *** No targets specified and no makefile found.  Stop.
make[1]: Leaving directory `/home/lotte/bachelor/riot/RIOT/pkg/oonf_api'
make: *** [/home/lotte/bachelor/riot/projects/olsr2/bin/native/oonf_api.a] Error 2

After I copied all oonf files from RIOT/pkg/oonf_api/oonf_api to RIOT/pkg/oonf_api, I got the following errors:

autobuf.c:60:10: fatal error: 'common/autobuf.h' file not found
#include "common/autobuf.h"
         ^
1 error generated.
make[2]: *** [/home/lotte/bachelor/riot/projects/olsr2/bin/native/autobuf.o] Error 1
make[2]: Leaving directory `/home/lotte/bachelor/riot/RIOT/pkg/oonf_api/src-api/common'
make[2]: Entering directory `/home/lotte/bachelor/riot/RIOT/pkg/oonf_api/src-api/rfc5444'
clang -DRIOT -DENABLE_NAME -ggdb -DBOARD_NATIVE -DINIT_ON_START -DBOARD=NATIVE -DCPU_NATIVE -DMODULE_AUTO_INIT -DMODULE_RTC -DMODULE_UART0 -DMODULE_POSIX -DMODULE_PS -DMODULE_SHELL -DMODULE_SHELL_COMMANDS -DMODULE_RANDOM -DMODULE_CONFIG -DMODULE_OLSR2 -DMODULE_NATIVENET -DMODULE_CPU -DMODULE_CORE -DMODULE_SYS -DMODULE_LIB -DMODULE_VTIMER -DMODULE_DESTINY -DMODULE_SIXLOWPAN -DMODULE_OONF_COMMON -DMODULE_OONF_RFC5444 -DMODULE_NET_HELP -DMODULE_IEEE802154 -DMODULE_SEMAPHORE -DMODULE_TRANSCEIVER -DMODULE_HWTIMER -DMODULE_TIMEX -Wall -Wextra -pedantic -m32 -std=gnu99 -DVERSION=\"2014.01-301-g56c34-pr/765\" -I/home/lotte/bachelor/riot/RIOT/sys/net/include -I/home/lotte/bachelor/riot/RIOT/core/include -I/home/lotte/bachelor/riot/RIOT/drivers/include -I/home/lotte/bachelor/riot/RIOT/sys/include -I/home/lotte/bachelor/riot/RIOT/boards/native/include -I/home/lotte/bachelor/riot/RIOT/cpu/native/include -c rfc5444.c -o /home/lotte/bachelor/riot/projects/olsr2/bin/native/rfc5444.o
rfc5444.c:43:10: fatal error: 'common/common_types.h' file not found
#include "common/common_types.h"
         ^
1 error generated.
make[2]: *** [/home/lotte/bachelor/riot/projects/olsr2/bin/native/rfc5444.o] Error 1
make[2]: Leaving directory `/home/lotte/bachelor/riot/RIOT/pkg/oonf_api/src-api/rfc5444'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/lotte/bachelor/riot/RIOT/pkg/oonf_api'
make: *** [/home/lotte/bachelor/riot/projects/olsr2/bin/native/oonf_api.a] Error 2

benpicco · 2014-02-24T19:34:57Z

des-testbed/RIOT-projects#98 depends on #765

Lotterleben · 2014-02-24T23:49:15Z

I know, that's why I made sure I checked it out in my RIOT folder before I tried to compile des-testbed/RIOT-projects#98. But you're right, this is not the right place for that comment (even though the double nesting of oonf_api/oonf_api struck me as odd).

benpicco · 2014-02-26T01:01:08Z

When I do

git clone [email protected]:benpicco/RIOT.git && cd RIOT
git checkout -b oonf_api origin/oonf_api
git checkout -b olsr2 origin/olsr2
git merge oonf_api

and

git clone [email protected]:benpicco/projects.git && cd projects
git checkout -b olsr2 origin/olsr2
ln -s /path/to/RIOT/ .

projects/olsr2 does build

(the example project only calls USEMODULE for olsr2, Makefile.dep takes care of including oonf_rfc5444, oonf_common and the network stack)

Lotterleben · 2014-02-27T13:16:19Z

Yup, when I do that, it builds on my computer as well. What I did was:
In RIOT/

git fetch origin
git checkout pr/765
git checkout -b oonftest2
git merge pr/630

in projects/

git fetch origin
git checkout pr/98

Lotterleben · 2014-03-03T15:39:29Z

@benpicco Ping? I'm really interested in resolving this one...

benpicco · 2014-03-04T04:12:09Z

Umm, are you sure that projects/RIOT is the same as the one where you merged the pr?

If I use the current upstream master and do

git fetch origin
git merge origin/pr/765 
git merge origin/pr/630

and

git fetch origin
git merge origin/pr/98

it builds just fine.

Lotterleben · 2014-03-04T22:48:17Z

I'm pretty sure I did, but yup, with a clean slate it works for me too. Sorry about the fuss.

Lotterleben · 2014-03-11T22:34:15Z

Seems to be working fine, if anyone wants to review & merge ;)

[RFC] add oonf_api to pkg

LudwigKnuepfer · 2014-03-14T08:29:02Z

@benpicco I created #884 for you but can not assign you (I guess that is because you are not part of the RIOT-OS orga..).

OlegHahm · 2014-03-15T10:05:47Z

Added @benpicco to the Github RIOT developer team and assigned the issue to him.

OlegHahm assigned Lotterleben Feb 6, 2014

OlegHahm added this to the Release NEXT MAJOR milestone Feb 6, 2014

benpicco mentioned this pull request Feb 12, 2014

Add USEPKG variable for Makefiles #576

Merged

add oonf_api pkg

edea31a

benpicco mentioned this pull request Feb 22, 2014

[RFC] add olsr2 routing algorithm #765

Closed

benpicco mentioned this pull request Feb 24, 2014

add olsr2 example application des-testbed/RIOT-projects#98

Open

LudwigKnuepfer added the enhancement label Mar 12, 2014

Lotterleben added a commit that referenced this pull request Mar 13, 2014

Merge pull request #630 from benpicco/oonf_api

2755b69

[RFC] add oonf_api to pkg

Lotterleben merged commit 2755b69 into RIOT-OS:master Mar 13, 2014

LudwigKnuepfer mentioned this pull request Mar 14, 2014

pkg: oonf_api: make patch not working #884

Closed

benpicco deleted the oonf_api branch March 17, 2014 15:39

OlegHahm mentioned this pull request Jan 13, 2015

routing: add OLSRv2 #2294

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC] add oonf_api to pkg #630

[RFC] add oonf_api to pkg #630

benpicco commented Feb 6, 2014

mehlis commented Feb 6, 2014

LudwigKnuepfer commented Feb 6, 2014

Lotterleben commented Feb 9, 2014

mehlis commented Feb 9, 2014

benpicco commented Feb 12, 2014

mehlis commented Feb 12, 2014

OlegHahm commented Feb 15, 2014

benpicco commented Feb 18, 2014

benpicco commented Feb 18, 2014

Lotterleben commented Feb 19, 2014

Lotterleben commented Feb 19, 2014

benpicco commented Feb 19, 2014

Lotterleben commented Feb 24, 2014

benpicco commented Feb 24, 2014

Lotterleben commented Feb 24, 2014

benpicco commented Feb 26, 2014

Lotterleben commented Feb 27, 2014

Lotterleben commented Mar 3, 2014

benpicco commented Mar 4, 2014

Lotterleben commented Mar 4, 2014

Lotterleben commented Mar 11, 2014

LudwigKnuepfer commented Mar 14, 2014

OlegHahm commented Mar 15, 2014

[RFC] add oonf_api to pkg #630

[RFC] add oonf_api to pkg #630

Conversation

benpicco commented Feb 6, 2014

mehlis commented Feb 6, 2014

LudwigKnuepfer commented Feb 6, 2014

Lotterleben commented Feb 9, 2014

mehlis commented Feb 9, 2014

benpicco commented Feb 12, 2014

mehlis commented Feb 12, 2014

OlegHahm commented Feb 15, 2014

benpicco commented Feb 18, 2014

benpicco commented Feb 18, 2014

Lotterleben commented Feb 19, 2014

Lotterleben commented Feb 19, 2014

benpicco commented Feb 19, 2014

Lotterleben commented Feb 24, 2014

benpicco commented Feb 24, 2014

Lotterleben commented Feb 24, 2014

benpicco commented Feb 26, 2014

Lotterleben commented Feb 27, 2014

Lotterleben commented Mar 3, 2014

benpicco commented Mar 4, 2014

Lotterleben commented Mar 4, 2014

Lotterleben commented Mar 11, 2014

LudwigKnuepfer commented Mar 14, 2014

OlegHahm commented Mar 15, 2014