Gentoo Development

Links and documentation

• Official handbook
• Ebuilds Notes
• Maintained Packages

System configuration

General setup

• Your ~/.cvsrc file should contain the following:

cvs -q -z0
diff -u -b -B
checkout -P
update -d -P

• You should add FEATURES="test" to your make.conf file. Note that the test USE flag on the ebuilds is only used to trigger additional dependencies that may be required for testing. This test USE flag is automatically on if you have FEATURES="test", thus you cannot turn off tests per package via /etc/portage/package.use. It is even hard to turn off tests per package, the only way is manually emerging with FEATURES="-test" the problematic package.

Repositories (Main, overlays and layman)

• To check out the main tree (still in CVS):

cvs -d elvanor@cvs.gentoo.org:/var/cvsroot co gentoo-x86

• To change the location of your Portage tree, to test out an ebuild you just modified:

PORTDIR="/home/elvanor/development/org.gentoo/gentoo-x86/" emerge -a eclipse-ecj:3.4 

• Overlays are additional ebuilds repertories in addition to the main, stable Gentoo tree. As such they are useful for development activities or hosting of unstable ebuilds.

• layman is a tool for overlay management. It uses a list of overlays. This list is stored somewhere (generally on a remote server). The main configuration file for layman is /etc/layman/layman.cfg.

• When adding URI for overlay lists in that file, be sure to add a space or tab before every new line! This is mandatory, else layman won't parse the file correctly. I personally consider this a bug...

• Unofficial overlays can be listed with the -k (nocheck) switch:

layman -L -k

Creating a private corporate overlay

• To secure the distribution files transfers, you need to use SSH. A possible way is to change the FETCHCOMMAND that Portage uses to fetch the files. By default Portage uses wget that does not support sftp / scp, however curl does:

FETCHCOMMAND="/usr/bin/curl -f -L -u root: --key /etc/portage/com.shoopz.distfiles_dsa --pubkey /etc/portage/com.shoopz.distfiles_dsa.pub -o
    \"\${DISTDIR}/\${FILE}\" \"\${URI}\"
RESUMECOMMAND="/usr/bin/curl -f -L -u root: --key /etc/portage/com.shoopz.distfiles_dsa --pubkey /etc/portage/com.shoopz.distfiles_dsa.pub -C - -o
    \"\${DISTDIR}/\${FILE}\" \"\${URI}\""

Note that it seems Portage uses the portage user when performing the fetch. Thus the key you will create to authorize distfiles transfers needs to be readable by the portage user.

• Note that with libssh2-1.2.2, the known_hosts implementation is buggy. First it does not support multiple hosts on a single line in the known_hosts file, and second, you cannot use the global known_hosts file in /etc/ssh/ssh_known_hosts. The -k switch turns off known_hosts checking.
• The -L option tells curl to follow redirects (HTTP 301, 302 status codes).
• You can then use scp:// scheme in your ebuilds for SRC_URI. You also need to set RESTRICT="mirror".

Useful tools

• pkg-config is a tool that is used to keep trace of the configuration and install options of packages. For example, it can gives you the C flags options, or the value of a specific variable (sdkdir for xulrunner). It creates *.pc files that are stored in /usr/lib/pkgconfig (on Gentoo). You query these files with the command-line pkg-config tool.

• Use repoman (which is included as part of Portage) as a QA tool. It will report basic errors in ebuilds. Adjust manually PORTDIR and PORTDIR_OVERLAY environment variables if need be.

• Currently you cannot use repoman commit with several directories, it will always act in the current directory.

General Ebuild development guide

• If you write a simple ebuild, you still have to place it in the main Portage tree or in an overlay to install it via Portage.

• Emerge app-portage/gentoolkit-dev which contains several utilities needed when developing on Gentoo.

• The PORTDIR environment variable controls the location of your Portage tree.

• Always use the program echangelog to document your changes. When you're using an overlay, use echangelog-tng (from the package overlay-utils). UPDATE: It seems echangelog-tng no longer exists and its functionality has been merged into echangelog. Set the environment variable ECHANGELOG_USER:

export ECHANGELOG_USER="Jean-Noël Rivasseau <elvanor@gmail.com>"

When creating or copying a new ebuild, the ChangeLog file should be deleted. Once echangelog is ran, it will be created and then can be added to CVS / SVN.

• Naming Rules: Use syntax like package-1.1_rc2. package-1.1-rc2 is an error. If you create a tarball from a Subversion checkout, name it package-1.1_pre1536 or package-1.1_pre20070628 (revision number or date).

• To clean the temporary work dir, or if anything goes strange, use ebuild clean command.

• To get the latest version of an installed package, you can use best_version. It will work somehow with SLOTs - you can specify stuff like

best_version=dev-java/package-2.5.6*

best_version is an helper function.

• All packages should specify a slot. Those that don't need slotting specify SLOT="0".

• For metadata.xml, use the skeleton in /etc/portage/skel.metadata.

Patches

• Patches must be kept in the files/ directory of the ebuild location if they are small. They should not be compressed.

• Create a patch in the following way:

diff -u original-file.c modified-file.c > packagename-1.2.5-file.patch

• Note that it is better to apply patches from the "${S}" directory. That means you have to create the patch in the top-level directory. One way to proceed too is to duplicate the whole source tree and create the patch with the -r (recursive) flag to diff. Note however that patches are a pain to maintain when doing version bumps. Especially if a patch is applied to a set of files. Consider using sed if you can.

• The -b option can also be useful; it will ignore any difference in white space and newlines. For internal projects this should not be used, but to send patches upstream sometimes it comes handy.

• Apply it like this (in an ebuild):

epatch "${FILESDIR}/${P}-file.patch"

You may need to cd to the correct directory if you did not create the patch in the top-level directory of the package. Also don't forget that you need to inherit eutils eclass.

• epatch will keep trying various patch levels until it succeeds. It does not output .rej files; you have to apply the patch yourself to see if it works or not.

• The appropriate function to perform patching is src_prepare.

Manifests

• ebuild myebuild-1.0.ebuild digest will create the manifest for you. It must be able to access the distribution file. If the SRC_URI does not work yet, put the distfile in /usr/portage/distfiles directly.

• The "--force" option will prevent ebuild from marking the local distfile as invalid if it does not match an existing manifest entry. Usually you should not use it, except if you are also upstream for the ebuild you are writing, and you just changed the distfile.

Variables

• A list of important variables available when writing ebuilds:
  • ${S}: Path to the temporary build directory.
  • ${D}: Path to the temporary install directory.
  • ${P}: Package name and version.
  • ${PV}: Package version.
  • ${A}: A list of all the source files. Note this one should NOT be quoted as it can represent several distinct files (repoman will complain on this).

• The RESTRICT variable can be useful too. In particular RESTRICT="mirror" prevents Portage from trying to fetch the file from common Gentoo mirrors. This can be useful for private ebuilds with restricted distribution files.
• RESTRICT="test" should be used to disable failing tests.

Versions

• The "*" symbol cannot be used with >= style syntax in DEPEND or RDEPEND; only with "=".
• SLOT dependencies (since EAPI 1) are possible with the ":" syntax.
• More information.

Phases

List in order (EAPI 2)

• pkg_setup
• src_unpack
• src_prepare
• src_configure
• src_compile
• src_test
• src_install
• pkg_preinst
• pkg_postinst
• pkg_prerm
• pkg_postrm

Patching and other preparations (src_prepare)

• Patching should be done in this phase (which is present since EAPI 2).

Configure Step (src_configure)

• If you use EAPI2 or later, you must call econf in src_configure, NOT in src_compile.
• There seems to be no way to remove the arguments that econf sends (eg, --build=). So if you need more control, the recommended way is to call configure manually.

Install Step (src_install)

• doins will NOT respect file permissions. You need to use doexe or dobin if you need to install executable files.
• doins will also act strangely with respect to empty directories. It may not create them. In this case you should use dodir or cp -r to copy a hierarchy of directories to the image install location.
• Changing permissions of files work normally using chmod.

Post-installation step (pkg_postinst)

• This phase can be useful if you need to perform actions after the files have been actually installed (written to the file system). Beware that it can be dangerous though, as a failure in this phase does not prevent the package from having been installed.

Helper Functions

• dohtml should be used to install HTML documentation files. Note it accepts an "-r" recursive option for installing a directory.

• dodoc always acts even if doc USE flag is not set, thus write something like:

use doc && dodoc doc/manual.pdf

Also note that currently dodoc does not have an -r option, but it's planned for EAPI-3.

• The functions to create new users or groups (enewuser, enewgroup) come from the eutils eclass. Don't forget to inherit this eclass if you use those functions!

USE flags

• As per GLEP-56, it is now mandatory to include a description of a (specific) USE flag in the metadata.xml file. Note that repoman will complain on this with something like: IUSE.invalid.

Concepts

• If a file collision occurs with two packages, it's almost always better to split the common files into a single package (even if it's only a single file). The alternatives are usually much worse. In particular, be careful, do NOT test the filesystem for the presence of the file, not emerging the file in that case! This can cause a reemerge of the package to lose that file (it will be correctly installed the first time since absent, but the second time it will be detected present, not installed but unmerged from the previous installation).

Static libraries, .la files

• Quite often the static libraries should not be build (econf --disable-static). Also many .la files are useless and should be removed in the src_install phase after emake install has been ran.

Profiles

• The best documentation to understand profiles is the PMS (Package Manager Specifications) right now. The drafts are available in PDF format.
• The new way of laying out profiles is arch/ia64. There is also default/linux/ia64. The default-linux directory seems to be deprecated.

Inheritance

• In a profile, the parent files reference other files that will add some data to the profile. For instance, /usr/portage/profiles/default/linux/x86/2008.0/desktop references /usr/portage/profiles/arch/x86, and even /usr/portage/profiles/base. Same for /usr/portage/profiles/releases and /usr/portage/profiles/targets.

Per package USE flag masking

• A globally masked USE flag can be enabled for a specific package by writing (here we take the real USE flag example):

media-libs/xine-lib -real

Note the "-".

Gentoo init scripts

• Official Documentation
• Writing basic init scripts in Gentoo is easy. Those scripts are ran via the /sbin/runscript.sh shell script. If you want to start a daemon, you should use the start-stop-daemon binary. It has options for starting a daemon, and kill older daemons via a pidfile and / or an executable name (see the man page).
• You cannot call exit() in your script. If the daemon you are launching kills the start-stop-daemon or runscript.sh processes, this will cause problems with Gentoo's init system.

start-stop-daemon

• Be very careful with s-s-d. In particular, even if you don't use the --exec argument explicitely, it still seems s-s-d will try to check for processes matching a particular name (maybe they take it from --start-as). So for instance if you launch a Python daemon, make sure your --exec argument points to the *exact* python interpreter the daemon is called with.
• If the script does not output [ok] but [!!], it usually means s-s-d was not able to match a pid (either via --exec or via --pid-file). s-s-d sends TERM signals to processes it started if they don't match what it expects (thus be careful about --exec as said before).
• s-s-d has an useful --chdir option that allows the started daemon to start in a given directory. Note that good practice says that daemon should start in /, but this is not always respected...

Order of execution

• With OpenRC, you can apparently modify the order of execution of the init scripts via their conf.d file, which is very convenient. With the old baselayout, there is no way to achieve this easily (eg, it's all done via the information present in the init scripts, which may not be exactly what you need in some cases).

Java Gentoo Development

• Some additional knowledge is required when writing Java-based ebuilds.