Intel Houdini is applied in x86 arch Android devices to enable the possibility of ARM code support. It's awesome and how about extending the possibility to Linux?
The Intel Houdini binaries is proprietary, you can only find them in several x86 model Android devices' vendor binaries. Nexus Player (fugu) contains currently the latest version that I can find.
https://developers.google.com/android/drivers#fuguopr2.170623.027
SHA256: 114c20b8335f3c166c76d590c6d238153c463c2fa2dea3da56d1ce3ffd1ec6ed
Accept the license at your own risk. Extract it and you get the Houdini and other unrelated binaries in vendor/intel/fugu/proprietary.
In vendor/intel/fugu/proprietary/device-partial.mk, files manifest is located here. All you need to do is copy them to your Linux host. However, the target path is Android style and this path style is hard-coded in Houdini binaries, so just take it easy, mkdir /system, try script below (root needed).
import os
import shutil
with open('device-partial.mk') as f:
for line in f:
line = line.strip()
if line.startswith(r"vendor/intel/fugu/"):
j = line.split(':')
if len(j) < 2:
continue
src = j[0]
src = src[src.find(r"proprietary/"):]
dst = '/'+j[1]
dst = os.path.dirname(dst)
if not os.path.exists(dst):
os.makedirs(dst)
shutil.copy(src, dst)
print("src %s copied to dst %s" % (src, dst))
Houdini relies binfmt_misc feature by Linux Kernel to pass the ARM executables to Houdini translator. You need to register ARM executable magic to Houdini translator /system/bin/houdini.
su
modprobe binfmt_misc
mount -t binfmt_misc none /proc/sys/fs/binfmt_misc
cd vendor/intel/fugu/proprietary
cat arm_dyn > /proc/sys/fs/binfmt_misc/register
cat arm_exe > /proc/sys/fs/binfmt_misc/register
Next step, install ARM runtime libraries on Linux. For Ubuntu 16.04:
# For default APT servers on Ubuntu x86 provide no ARM support, you need to add restriction to fetch only i386, amd64 packages from main server. Example: deb [arch=i386,amd64] http://archive.ubuntu.com/ubuntu/ xenial main restricted
# Then add APT servers providing ARM packages.
deb [arch=armhf] http://ports.ubuntu.com/ubuntu-ports/ xenial main
deb [arch=armhf] http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main
deb [arch=armhf] http://ports.ubuntu.com/ubuntu-ports/ xenial-security main
apt update
apt install libc6:armhf
Now we need Houdini using Linux GNU linker instead of Android linker. And chmod.
su
cp /lib/arm-linux-gnueabihf/ld-2.23.so /system/lib/arm/linker
chmod 755 /system/lib/arm/linker
chmod 755 /system/bin/houdini
Almost done. Let's try run OpenSSL for benchmark. Firstly, download a ARMHF openssl:
apt install libssl1.0.0:armhf
# DO NOT apt install openssl:armhf for breaking many things.
# Extract openssl from http://ports.ubuntu.com/ubuntu-ports/pool/main/o/openssl/openssl_1.0.2g-1ubuntu4.10_armhf.deb
Here is the result:
# NATIVE
~/poc$ openssl speed -evp AES256
Doing aes-256-cbc for 3s on 16 size blocks: 90227643 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 64 size blocks: 24112794 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 256 size blocks: 6133873 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 1024 size blocks: 1526964 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 8192 size blocks: 192347 aes-256-cbc's in 3.00s
OpenSSL 1.0.2g 1 Mar 2016
built on: reproducible build, date unspecified
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx)
compiler: cc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 481214.10k 514406.27k 523423.83k 521203.71k 525235.54k
# ARMHF
~/poc$ ./openssl speed -evp AES256
ERROR: ld.so: object '/system/lib/arm/libaeabi_map.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
Doing aes-256-cbc for 3s on 16 size blocks: 10090430 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 64 size blocks: 3350791 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 256 size blocks: 878897 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 1024 size blocks: 176189 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 8192 size blocks: 27901 aes-256-cbc's in 3.00s
OpenSSL 1.0.2g 1 Mar 2016
built on: reproducible build, date unspecified
options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) blowfish(ptr)
compiler: cc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 53815.63k 71483.54k 74999.21k 60139.18k 76188.33k
Additional, to fix '/system/lib/arm/libaeabi_map.so' from LD_PRELOAD warning.
The cause is that Houdini will set LD_PRELOAD=/system/lib/arm/libaeabi_map.so, libaeabi_map.so is not neccesary for us as it's used for patching libc.so and libm.so on Android.
So, NUL the path can silent the warning.
su
sed -i 's/\/system\/lib\/arm\/libaeabi_map\.so/\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0/g' /system/bin/houdini
Bugs may exist for this Houdini binaries was not intended for Linux.
