[Android] Developing Android-native-library on 64bit build machine

Most Android device is 32bit machine. So, many application assumes that host machine is 32bit system.
And in general, there is no difference developing Android-natvie-library between 64bit and 32bit build machine.

But, here is usual step for developing Android-native-library.
(1) Developing and verifying code at build machine.
(2) Porting to NDK build system.

Most developers turns on full-warning-option at compile to detect bugs at early stage.
But, building and verifying codes assuming 32bit host machine at 64bit machine always issues type casting warning due to different type size.
Especially, between pointer and integer.

For example, many Android JAVA application uses intjint – as a type to contain native pointer with assumption of 32bit-host-system.
Building this code at 64bit build system to verify code issues type casting warning, even if code itself is built perfectly at NDK build system.
And it is worse that this code doesn’t work at 64bit Android host machine, even though it is not popular.

To reduce this warnings (for easy-verifying of library code at build machine), in my opinion, using longjlong – instead of jint as a type for containing native pointer is better unless memory space is extremely critical.
And to make compiler be happy, using macro can be a good choice.
Here is example of macro for type-casting between pointer and integer – jlong.
(This sample works well without warning at 32bit/64bit build/host system).

#define ptr2jlong(v) ((jlong)((intptr_t)(v)))
#define jlong2ptr(v) ((void*)((intptr_t)(v)))

This is just simple example for portability issues.
Making portable code is always very difficult…

[ARM] Multi-core optimzation test…

This test is done under following environment.

x86 :

intel Core(TM)2 Duo T9400 2.53Mhz
GCC-4.4.5

ARM :

OMAP4430 (Cortax-A9)
Android NDK platform 9

Test code.

/*
 * Test Configuration
 */
#define _ARRSZ    1024*1024*8

static int _arr[_ARRSZ];

static void
_init() {
    int i;
    for (i = 0; i < _ARRSZ; i++)
        _arr[i] = i;
}

static unsigned long long
_utime() {
    struct timeval tv;
    if (gettimeofday(&tv, NULL))
        assert(0);
    return (unsigned long long)(tv.tv_sec * 1000000)
        + (unsigned long long)tv.tv_usec;
}

#define _test_arraycopy_pre()               \
    int  i;                                 \
    unsigned long long ut;                  \
    int* ia = malloc(_ARRSZ * sizeof(*ia));

#define _test_arraycopy_post()              \
    free(ia);

#define _operation()            \
    do {                        \
        ia[i] = _arr[i];        \
    } while (0)

static void*
_test_arraycopy_worker(void* arg) {
    int     i;
    int*    ia = arg;
    for (i = (_ARRSZ / 2); i < _ARRSZ; i++)
        _operation();
    return NULL;
}

static unsigned long long
_test_arraycopy_sc() {
    _test_arraycopy_pre();

    ut = _utime();
    for (i = 0; i < _ARRSZ; i++)
        _operation();
    ut = _utime() - ut;

    _test_arraycopy_post();

    return ut;
}

static unsigned long long
_test_arraycopy_dc() {
    pthread_t thd;
    void*     ret;
    _test_arraycopy_pre();

    ut = _utime();
    if (pthread_create(&thd,
               NULL,
               &_test_arraycopy_worker,
               (void*)ia))
        assert(0);

    for (i = 0; i < (_ARRSZ / 2); i++)
        _operation();

    if (pthread_join(thd, &ret))
        assert(0);

    ut = _utime() - ut;

    _test_arraycopy_post();

    return ut;
}

#undef _test_arraycopy_pre
#undef _test_arraycopy_post

int
main(int argc, char* argv[]) {
    _init();
    printf(">> SC : %lld ", _test_arraycopy_sc());
    printf(">> DC : %lld\n", _test_arraycopy_dc());
    return 0;
}

[Test 1]
x86

>> SC : 59346 >> DC : 38566
>> SC : 59195 >> DC : 39028
>> SC : 49529 >> DC : 38160
>> SC : 49722 >> DC : 38457
>> SC : 49952 >> DC : 37457

ARM

>> SC : 102295 >> DC : 94147
>> SC : 102264 >> DC : 94025
>> SC : 102173 >> DC : 94116
>> SC : 102172 >> DC : 94116
>> SC : 102325 >> DC : 94177

Change ‘_operation’ macro to as follows

#define _operation()                                    \
    do {                                                \
        if (i > _ARRSZ / 2)                             \
            ia[i] = (_arr[i] & 0xff) << 8 ^ _arr[i];    \
        else                                            \
            ia[i] = (_arr[i] & 0xff) << 16 ^ _arr[i];   \
    } while (0)                                         \

[Test 2]
x86

>> SC : 60696 >> DC : 40523
>> SC : 56907 >> DC : 45355
>> SC : 55066 >> DC : 42329
>> SC : 54931 >> DC : 40651
>> SC : 57022 >> DC : 41879

ARM

>> SC : 164514 >> DC : 112671
>> SC : 163971 >> DC : 112854
>> SC : 164521 >> DC : 112976
>> SC : 163940 >> DC : 112732
>> SC : 164245 >> DC : 112671

Interesting result, isn’t it?
For heavily-memory-accessing-code (Test 1), ARM does not show good statistics for multi-core (in this case, dual-core) optimization.
But, if not (Test 2), optimization shows quite good results.
And, x86 seems to handle memory accessing from multi-core, quite well.

So, developers should consider ARM’s characteristic when optimize codes for multi-core.
(I’m sure that ARM will improve this someday! :-) )

* Things to consider regarding this kind of optimization *
Cache, Cache coherence, Memory Controller, Bus etc…

[ Test for later version of ARM (ex Cortax-A15) will be listed continuously… ]

[Android] NDK issues (found)

* common

+ '-rdynamic' is not supported.
  android dynamic linker allows 'undefined symbol' only for weak symbol.(See (*1))
+ 'tmpfile()' function always returns NULL. "No such file or directory!"
+ 'strtod()' doesn't support '0xXXXX' format.

* NDK 8

'regex.h' exists in 'include'. But function bodies are not in library.

* NDK 9

'pthread_rwlock_xxxx' is declared in 'pthread.h'. But body is missing in library.
'pthread_exit' is declared without 'noreturn' attribute.
'regex' functions doesn't work as it should be. (bug? or not-implemented yet?? I have no idea.)

to be continued…

===== Details =====

(*1) : ‘-rdynamic’ and ‘weak’ symbol (Updated at 2011-Aug-29)

Here is sample code to for testing ‘weak’ symbol at Android.

[ main.c ]
#include <dlfcn.h>
#include <stdio.h>

int g_d;

void
main() {
    void* handle;
    void  (*prf)(void);

    g_d = 369;

    handle = dlopen("/data/libgtst.so", RTLD_LAZY);
    if (!handle) {
        printf("Fail to load library\n");
        return;
    }

    prf = dlsym(handle, "print_g");
    if (NULL != dlerror()) {
        printf("Fail to get symbol print_g\n");
        return;
    }

    (*prf)();
    printf("=== DONE! ===\n");
}

[ lib.c ]
#include <stdio.h>

extern int g_d __attribute__((weak));

void
print_g(void) {
    printf("==> %d\n", g_d);
}

[ Android.mk ]
LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)
LOCAL_MODULE := libgtst
LOCAL_SRC_FILES := lib.c

LOCAL_CFLAGS :=
LOCAL_C_INCLUDES += $(NDK_PROJECT_PATH)
include $(BUILD_SHARED_LIBRARY)

include $(CLEAR_VARS)
LOCAL_MODULE := test
LOCAL_SRC_FILES := main.c
LOCAL_CFLAGS :=
LOCAL_LDFLAGS :=
LOCAL_C_INCLUDES += $(NDK_PROJECT_PATH)
include $(BUILD_EXECUTABLE)

----- Test Steps -----
* build above codes at Android NDK
adb push libgtst.so /data/libgtst.so
adb push test /data/test
adb shell /data/test
==> 369
=== DONE! ===

to be continued…