Cross-compiling in chroot environment
About this tutorial:
This tutorial will show you step by step the installation of a chroot environment, which is used to cross-compile the linux kernel and cross-emerge the whole system.The chroot environment is set up in order to be independent from the main root folder and protect it. Because cross-compilation or cross-emerge need specific overlays, package versions or configurations, it is better to use a separate chroot environment for it.
For some problems which occured, simple workarounds are presented. Any hints or improvements are welcome.
Last time the commands have been tested: 2010.09.15
Important docs or information:
http://gentoo.mindzoo.de/index.cgi/wiki/cross-compiling
http://www.gentoo.org/proj/en/base/embedded/handbook/
http://www.gentoo.org/proj/en/overlays/userguide.xml
http://git.overlays.gentoo.org/gitweb/?p=proj/embedded-cross.git;a=summary
Prepare the installation
Login as root
Check the current settings
emerge --info
Set path macros. Here: use root folder
unset ROOT unset PORTAGE_CONFIGROOT
Prepare etc/portage folder
cd $PORTAGE_CONFIGROOT/etc/portage
Which package.* are still files.
ls -l package.*
Replace package.* files by folders
mv package.keywords local-keywords mv package.mask local-mask mv package.use local-use mkdir -p $PORTAGE_CONFIGROOT/etc/portage/package.keywords mkdir -p $PORTAGE_CONFIGROOT/etc/portage/package.mask mkdir -p $PORTAGE_CONFIGROOT/etc/portage/package.use mv local-keywords package.keywords mv local-mask package.mask mv local-use package.use
Install layman on your system
Define use flags for layman
cd $PORTAGE_CONFIGROOT/etc/portage/package.use nano layman-use
New or changed content of file layman-use:
app-portage/layman bazaar git subversion dev-vcs/git bash-completion cvs doc subversion tk dev-vcs/subversion bash-completion ctypes-python doc extras -dso dev-db/sqlite extensions
Emerge layman
emerge --ask -v layman
Workaround for problem with dev-perl/DBD-SQLite-1.29-r2
This problem has been encountered on 2010.09.15
When you emerge layman you could get the error message:
Output:
emerge: there are no ebuilds built with USE flags to satisfy ">=dev-db/sqlite-3.6.22[extensions]".!!! One of the following packages is required to complete your request: - dev-db/sqlite-3.6.23.1 (Change USE: +extensions)
This problem occurs even when USE flag extensions is defined for sqlite.
This problem occurs when dev-perl/DBD-SQLite-1.29-r2 is emerged.
This problem doesn't occur when dev-perl/DBD-SQLite-1.29 or dev-perl/DBD-SQLite-1.14 are emerged.
It seems that /usr/portage/dev-perl/DBD-SQLite/DBD-SQLite-1.29-r2.ebuild is missing inheritance from eutils.
Solution:
Mask this version of DBD-SQLite.
nano $PORTAGE_CONFIGROOT/etc/portage/package.mask/layman-mask
New or changed content of file layman-mask:
# because USE flag "extensions" is not found for dev-db/sqlite =dev-perl/DBD-SQLite-1.29-r2
Try to emerge layman again.
Configure layman
Prepare make.conf for layman, but don't use it here
nano $PORTAGE_CONFIGROOT/etc/make.conf
New or changed content of file make.conf:
# layman # source /var/lib/layman/make.conf # crossdev uses the first overlay PORTDIR_OVERLAY="/usr/local/portage $PORTDIR_OVERLAY"
Load overlay embedded-cross
layman -a embedded-cross
Create host system for cross-compilation
Set path macros. Here: don't use your real root folder!
/MyDevelop/host-openmoko is the folder where the new root file system for chroot is prepared.
export ROOT=/MyDevelop/host-openmoko export PORTAGE_CONFIGROOT=/MyDevelop/config/host-openmoko
Create root directory for new linux root file system
mkdir -p $ROOT
Create portage config directory
mkdir -p $PORTAGE_CONFIGROOT/etc ln -s /usr/portage/profiles/default/linux/x86/10.0 $PORTAGE_CONFIGROOT/etc/make.profile cp /etc/make.conf $PORTAGE_CONFIGROOT/etc/make.conf nano $PORTAGE_CONFIGROOT/etc/make.conf
Change the make.conf file.
Keep your settings of
CFLAGS CXXFLAGS CHOST GENTOO_MIRRORS MAKEOPTS VIDEO_CARDS INPUT_DEVICES LINGUAS ACCEPT_LICENSE
New or changed content of file make.conf:
# define ROOT
ROOT="/MyDevelop/host-openmoko"
PKGDIR=${ROOT}/packages/
PORTAGE_TMPDIR=${ROOT}/tmp/
PORT_LOGDIR=${ROOT}/var/log/portage
PKG_CONFIG_PATH=${ROOT}/usr/lib/pkgconfig/
# reduce the USE flags to a minimum
USE="mmx sse sse2 symlink"
FEATURES="sandbox noman noinfo nodoc"
# PORTDIR_OVERLAY for layman
source /var/lib/layman/make.conf
# crossdev uses the first overlay
PORTDIR_OVERLAY="/usr/local/portage $PORTDIR_OVERLAY"
Prepare etc/portage folder
mkdir -p $PORTAGE_CONFIGROOT/etc/portage/package.keywords mkdir -p $PORTAGE_CONFIGROOT/etc/portage/package.mask mkdir -p $PORTAGE_CONFIGROOT/etc/portage/package.use
Which links should be created
ls -l /var/lib/layman/embedded-cross/portage-flags-host
Create links
ln -s /var/lib/layman/embedded-cross/portage-flags-host/keywords $PORTAGE_CONFIGROOT/etc/portage/package.keywords/openmoko-host-keywords
nano $PORTAGE_CONFIGROOT/etc/portage/package.mask/cross-host-mask
New or changed content of file cross-host-mask:
# use embedded-cross overlay >sys-devel/libtool-2.2.6b >dev-lang/python-2.6.4-r1
Check current settings. Folder tmp, var are created.
emerge --info -v
Check which packages would be installed
emerge -pv system
Check which packages would be installed from the embedded-cross overlay
emerge -pv system | grep "\[1\]"
Emerge system, e.g. gcc, python and portage are installed.
emerge --ask -v system
Install pkgconfig and setuptools, if they have not yet been installed
emerge --ask -v pkgconfig setuptools
Install layman on your system
Define use flags for layman
cd $PORTAGE_CONFIGROOT/etc/portage/package.use nano layman-use
New or changed content of file layman-use:
app-portage/layman bazaar git subversion dev-vcs/git bash-completion cvs doc subversion tk dev-vcs/subversion bash-completion ctypes-python doc extras -dso dev-db/sqlite extensions
Emerge layman
emerge --ask -v layman
Prepare host for chroot into build system
only once
mkdir $ROOT/etc cp $PORTAGE_CONFIGROOT/etc/make.conf $ROOT/etc cp -a $PORTAGE_CONFIGROOT/etc/portage $ROOT/etc ln -s /usr/portage/profiles/default/linux/x86/10.0 $ROOT/etc/make.profile nano $ROOT/etc/make.conf
New or changed content of file make.conf:
ROOT="" # cross compiling on CBUILD="i686-pc-linux-gnu"
Copy over nameserver information
cp -L /etc/resolv.conf $ROOT/etc/resolv.conf
Create folders, if they don't exist
ls $ROOT mkdir $ROOT/proc mkdir $ROOT/dev
Create folders
mkdir $ROOT/usr/portage mkdir -p $ROOT/usr/local/portage
Prepare host for chroot into build system
Each time you want to use chroot.
export ROOT=/MyDevelop/host-openmoko mount -t proc none $ROOT/proc mount -o bind /dev $ROOT/dev mount -o bind /dev/pts $ROOT/dev/pts mount -o bind /MyPortage/usr/portage $ROOT/usr/portage mount -o bind /MyPortage/usr/local/portage $ROOT/usr/local/portage
/MyPortage/usr/portage is the location of the real Portage tree in your real root file system. Usually this is /usr/portage.
/MyPortage/usr/local/portage is the location of your first portage overlay in your root file system. Usually this is /usr/local/portage.
Change root to the build system
chroot $ROOT /bin/bash
Set path macros, here: use root folder
unset ROOT PORTAGE_CONFIGROOT
You are now in your build system. Your real root is protected now.
. /etc/profile env-update
Define prompt
For you are logged in as root, you should know if you are in chroot or not.
export PS1="chroot \\[\\033[01;31m\\]\\u@\\h\\[\\033[01;34m\\] \\w \\$\\[\\033[00m\\] "
Check what is already installed
emerge -pv system
Create locale
nano -w /etc/locale.gen locale-gen
Disable layman/make.conf in make.conf
nano $PORTAGE_CONFIGROOT/etc/make.conf
New or changed content of file make.conf:
# layman #source /var/lib/layman/make.conf
List the packages in layman
layman -L
Load overlay embedded-cross
layman -a embedded-cross
Enable layman/make.conf in make.conf
nano $PORTAGE_CONFIGROOT/etc/make.conf
New or changed content of file make.conf:
# layman source /var/lib/layman/make.conf
Emerge and build cross compiler
Set keywords
nano $ROOT/etc/portage/package.keywords/crossdev-keywords
New or changed content of file cross-keywords:
# use the latest version of crossdev sys-devel/crossdev ~x86
Emerge crossdev
emerge --ask -v crossdev
Check which package versions crossdev will use
crossdev -P -pv --binutils 2.18-r4 --target armv4tl-softfloat-linux-gnueabi
Check if overlay is used
cross-armv4tl-softfloat-linux-gnueabi/binutils-2.18-r4
cross-armv4tl-softfloat-linux-gnueabi/gcc-4.4.3
cross-armv4tl-softfloat-linux-gnueabi/linux-headers-2.6.35
cross-armv4tl-softfloat-linux-gnueabi/glibc-2.11.2
Create cross-compiler
crossdev --binutils 2.18-r4 --target armv4tl-softfloat-linux-gnueabi
Check cross-emerge settings
crossdev calls emerge-wrapper. emerge-wrapper copies the folder /usr/share/crossdev/etc into /usr/armv4tl-softfloat-linux-gnueabi.
ls -l /usr/armv4tl-softfloat-linux-gnueabi/etc
Adjust the contents of make.conf
nano /usr/armv4tl-softfloat-linux-gnueabi/etc/make.conf
New or changed content of file make.conf:
CHOST=armv4tl-softfloat-linux-gnueabi
Check cross-emerge info
armv4tl-softfloat-linux-gnueabi-emerge --info -v
Check if SYSROOT=/usr/armv4tl-softfloat-linux-gnueabi
Check ROOT, PORTAGE_CONFIGROOT
Default settings of compilers are in
ls /usr/share/crossdev/include/site
Show installed overlays
layman -l
Which ebuilds are available
find /var/lib/layman/embedded-cross -name *.ebuild
Prepare usage of cross-compiler
Each time you want to use it.
export ROOT=/usr/armv4tl-softfloat-linux-gnueabi export PORTAGE_CONFIGROOT=/usr/armv4tl-softfloat-linux-gnueabi
Create etc folder
mkdir -p $PORTAGE_CONFIGROOT/etc
Prepare etc/portage folder
mkdir -p $PORTAGE_CONFIGROOT/etc/portage/package.keywords mkdir -p $PORTAGE_CONFIGROOT/etc/portage/package.mask mkdir -p $PORTAGE_CONFIGROOT/etc/portage/package.use
Edit make.conf and replace it completely by the text below
nano $PORTAGE_CONFIGROOT/etc/make.conf
New or changed content of file make.conf:
CHOST=armv4tl-softfloat-linux-gnueabi
# cross compiling on
CBUILD="i686-pc-linux-gnu"
ARCH=arm
E_MACHINE=EM_ARM
ACCEPT_KEYWORDS="arm" # Add ~arm here if you know what you do.
MARCH_TUNE="-march=armv4t -mtune=arm920t"
CFLAGS="-Os -pipe ${MARCH_TUNE} -fomit-frame-pointer"
CXXFLAGS="${CFLAGS}"
ROOT=/usr/${CHOST}/
PKG_CONFIG_PATH="${ROOT}/usr/lib/pkgconfig/"
#PORTDIR_OVERLAY="$location/openmoko-target ${ROOT}/usr/local/portage"
source /var/lib/layman/make.conf
VIDEO_CARDS="fbdev glamo dummy"
INPUT_DEVICES="evdev keyboard mouse tslib"
LINGUAS="en"
PKGDIR=${ROOT}/packages/
PORTAGE_TMPDIR=${ROOT}/tmp/
PORT_LOGDIR=${ROOT}/var/log/portage
PKG_CONFIG_PATH=${ROOT}/usr/lib/pkgconfig/
Add also the definitions of other macros to this file, like MAKEOPTS and GENTOO_MIRRORS.
Define make.profile and check if it points there.
cd $PORTAGE_CONFIGROOT/etc rm make.profile ln -s /var/lib/layman/embedded-cross/profiles/openmoko make.profile ls -l
Which links should be created
ls -l /var/lib/layman/embedded-cross/portage-flags-target
Create links
ln -s /var/lib/layman/embedded-cross/portage-flags-target/keywords $PORTAGE_CONFIGROOT/etc/portage/package.keywords/openmoko-keywords ln -s /var/lib/layman/embedded-cross/portage-flags-target/use $PORTAGE_CONFIGROOT/etc/portage/package.use/openmoko-use
Edit file openmoko-local-mask
nano $PORTAGE_CONFIGROOT/etc/portage/package.mask/openmoko-local-mask
New or changed content of file openmoko-local-mask:
# error in cross compilation, no overlay =app-arch/cpio-2.11 # use embedded-cross overlay >dev-lang/python-2.6.4-r1 >sys-devel/libtool-2.2.6b
Now you can build the kernel or the root file system. You may skip building the kernel and start with building the root file system
Build kernel
IMPORTANT: read how to cross compile the linux kernel:
http://www.gentoo.org/proj/en/base/embedded/handbook/?part=1&chap=4
Emerge kernel sources:
USE="symlink" armv4tl-softfloat-linux-gnueabi-emerge --nodeps --ask -v sys-kernel/gentoo-sources
Define macro
export SYSROOT=/usr/armv4tl-softfloat-linux-gnueabi
Build kernel and modules, install modules
cd ${SYSROOT}/usr/src/linux
make ARCH="arm" CROSS_COMPILE="armv4tl-softfloat-linux-gnueabi-" INSTALL_MOD_PATH="${SYSROOT}" help
make ARCH="arm" CROSS_COMPILE="armv4tl-softfloat-linux-gnueabi-" INSTALL_MOD_PATH="${SYSROOT}" menuconfig
make ARCH="arm" CROSS_COMPILE="armv4tl-softfloat-linux-gnueabi-" INSTALL_MOD_PATH="${SYSROOT}" vmlinux modules
make ARCH="arm" CROSS_COMPILE="armv4tl-softfloat-linux-gnueabi-" INSTALL_MOD_PATH="${SYSROOT}" modules_install
Build root file system
armv4tl-softfloat-linux-gnueabi-emerge --ask -v system
Workaround for problem with mpfr
Still needed on 2010.09.15.
When dev-libs/mpfr is built you get the error message:
Output:
/usr/lib/libgmp.so: could not read symbols: File in wrong format
Check the file type
ls -l /usr/lib/libgmp.so file /usr/lib/libgmp.so.3.5.2
Output:
/usr/lib/libgmp.so.3.5.2: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped
Check the file type
file /usr/armv4tl-softfloat-linux-gnueabi/usr/lib/libgmp.so.3.5.2
Output:
/usr/armv4tl-softfloat-linux-gnueabi/usr/lib/libgmp.so.3.5.2: ELF 32-bit LSB shared object, ARM, version 1 (SYSV), dynamically linked, stripped
/usr/armv4tl-softfloat-linux-gnueabi/usr/lib/libgmp.so is the library which should be linked
Change the libdir path in the libgmp.la file
nano /usr/armv4tl-softfloat-linux-gnueabi/usr/lib/libgmp.la
New or changed content of file libgmp.la:
libdir='/usr/armv4tl-softfloat-linux-gnueabi/usr/lib'
Resume
emerge --resume
Workaround for problem with shadow
Still needed on 2010.09.15.
When sys-apps/shadow is built you get the error message:
Output:
/usr/libexec/gcc/armv4tl-softfloat-linux-gnueabi/ld: skipping incompatible /lib/libcrack.so.2 when searching for /lib/libcrack.so.2 /usr/libexec/gcc/armv4tl-softfloat-linux-gnueabi/ld: cannot find /lib/libcrack.so.2
Change path in the libcrack.so file
nano $ROOT/usr/lib/libcrack.so
New or changed content of file libcrack.so:
GROUP ( /usr/armv4tl-softfloat-linux-gnueabi/lib/libcrack.so.2 )
Resume
emerge --resume
Installation and tests
No openmoko hardware has been available for installation and testing.
A similar sequence of commands has been used to cross-compile a root file system and a kernel for another ARM board and test it there.
Other usefull information
http://wiki.openmoko.org/wiki/Main_Page
http://neuvoo.org/wiki/index.php?title=Main_Page
http://gitorious.org/gentoo-arm-overlay
http://git.overlays.gentoo.org/gitweb/
