Bundle Buildsystem

From The Chakra Project

Català ─ Deutsch ─ Ελληνικά ─ English ─ Español ─ Euskara ─ فارسی ─ Français ─ Galego ─ Italiano ─ Polski ─ Português do Brasil ─ Svenska ─ Русский ─ O'zbek ─ ‪中文(简体)‬ ─ ‪中文(繁體)‬

Features

Chrooted build environment.
Clean and separated. No changes on the build machine are needed, just one new directory will be created.
Transparent creation of separate packages and bundles.
Does not run only on Arch but every Linux distro that has pacman.static installed.
Automatic chroot cleaning and bundle creation.
Does not break functionality. All standard makepkg stuff works, like repackaging for example.
Automatic repo handling (cleaning/moving packages, atomic uploads).
Install/reinstall/uninstall complete chroots with just one command.
A nice setup to get started in almost no time.

Before you start

Have in mind that some bundles might need to be compiled in a computer with the same architecture as the buildroot. Also, note you'll need about 500mb for the chroot install, plus whatever space the build/depends for your bundle require. That can be as much as 20Gb, for large bundles like libre-office. Make sure git is installed on the host system:

sudo pacman -Sy git

How to setup the bundles chroot

First, you'll need the chroot setup script:

wget http://gitorious.org/chakra-packages/buildsystem/blobs/raw/master/scripts/c-setup.sh

To make it executable:

[user@buildroot 32b]$ chmod +x c-setup.sh

Bundles don't need to be build for testing or stable branch separately, so only the architecture is a variable(i686/x86_64), "master" is the only option for repo variable.

Examples:

To create a x64 chroot:

./c-setup.sh bundles master x86_64

To create a x32 chroot (note the "n" switch for non-committer):

./c-setup.sh bundles master i686 n

(n)ewb-mode (optional 3rd argument) / (n)on-commiter mode means the chroot will not be setup for git commit access, but bundles will still be buildable. So if you are new to the bundle-creation game, start out using 'n', then upload the bundles for others to test.

Also, x86_64 = x64 and i686 = x32, so you can use them in the args as either.

Next the script will ask for your git access credentials, if you started the setup without the "n" switch. All other steps for the bundle chroot setup are identical to a regular package-chroot. Meaning installing the minimum packages needed to have a buildsystem, setting up password for the chroot, username, and in case of a committer, rsync access name and password.

When all is done, you'll be presented with this to enter the chroot:

:: Finally open _buildscripts/bundles-i686-makepkg.conf
:: and edit the DLAGENTS, CFLAGS, and CXXFLAGS settings to your
:: liking and you are ready to build packages.
 
:: (Very) Quick Start:
:: -------------------
:: 1 -> cd buildroot
:: 2 -> ./enter-chroot.sh bundles-i686
:: 3 -> cd package
:: 4 -> ../makepkg

Your first bundle

Now, that you have your chroot ready, it's time to build your first bundle to see how the system works. We have some bundles ready for building and testing, so just use the script makebundle like this:

./makebundle <bundle>

This builds or gets the package from the regular repo's for every library and binary the bundle needs to run. All this info is specified in the specifics bundle PKGINFO entry. All the packages created are saved in _pkgz and while the bundle is being created, all these packages are transferred to _temp. The default order for how the system looks for any needed package is:

Is it available in: _pkgz/_temp regular repo a PKGBUILD available in the bundle system for a package specified in PKGINFO

There are situation a bundle needs a newer version of a package then is available in the regular repo's, in those instances, there is a "@" switch, that will force the build of any needed package, regardless if it is available in _pkgz or regular repo.

To clarify all this some, one example of a PKGINFO of a bundle:

name=chrome-dev
desktop=chrome-dev.desktop
icon=chrome-dev.png
config=external
binary=
pkgz=gcc-libs libidl2 orbit2 atk libtasn1 gnutls libcups gdk-pixbuf2 gtk2 nss gconf libjpeg6 
freetype2 cairo alsa-lib libxss hicolor-icon-theme libjpeg perl libpng12 flashplugin-square 
chrome-dev
build=gcc-libs libidl2 orbit2 atk libtasn1 gnutls libcups gdk-pixbuf2 @gtk2 nss gconf libjpeg6
freetype2 cairo  alsa-lib libxss hicolor-icon-theme libjpeg perl libpng12 
flashplugin-square chrome-dev
env==LD_LIBRARY_PATH=$ORIGIN/lib:$ORIGIN/usr/lib:
$ORIGIN/usr/lib/gtk-2.0:$ORIGIN/opt/google/chrome/lib:$ORIGIN/usr/lib/mozilla/plugins
exec=/opt/google/chrome/google-chrome

name= refers to the PKGBUILD for that bundles, desktop= and icon= to those specific files inside the bundle directory. config= and binary= are variables to help with running and detecting the bundle for cbundle/cinstall. Next 2 lines are the main lines to really setup a bundle. pkgz= states every applications/package that will be included in the final bundle, and will run when a user clicks on a .cb file (.cb is the file extension used for bundles). build= show every packages that needs to be available for a succesful build of any package included in the pkgz line. In this example gtk2 has the "@" switch, so the newer version that is available in the bundle-system will be build. Also note, that in this example, every dep of gtk2 is included in the pkgz line. Some bundles run fine with only including the depencies listed in the PKGBUILD of a bundle, some bundles need all the the dependencies of such dep included in the pkgz= line.

env== line states some needed internal links for the bundle system to correctly find the needed .so files exec= line tells the bundle-system where to find the executable for that bundle.

When a bundle is succesfully created, it will be located in _repo/local. In case you are a commiter, you can upload it to the official repository running:

./upload.sh

To test it, just copy the created bundle in your system in any folder, click it and enjoy!

Now lets see an example in detail:

user@[chroot]:~/bundles$ ./makebundle  
 
>>> Makebundle 0.0.6
 
 :: Checking configs
 :: bundles-x86_64-cfg.conf OK 
 :: bundles-x86_64-makepkg.conf OK 
 :: user.conf OK 
 :: Loading configs
 :: you need to specify a bundle to build!
 :: available bundles:

amule
chrome-dev
chromium
~
xchat
xsane

These are the automated bundles currently available for testing. You can select any of these, in this example we are choosing filezilla:

manutortosa@[chroot]:~/buildroot/bundles$ ./makebundle filezilla

>>> bundle(r)

>> Checking configs
:: bundles-x86_64-cfg.conf OK 
:: bundles-x86_64-makepkg.conf OK 
:: user.conf OK 
>> Loading configs
:: building libgnomecups. makepkg flags: f
..

Here, the system starts the build of the needed packages, after this step the bundle is created.

>> Creating bundle: filezilla-3.3.3-3-x86_64 

:: Extracting xz package wxgtk...
:: Extracting xz package filezilla...
Parallel mksquashfs: Using 2 processors 
Creating 4.0 filesystem on filezilla-3.3.3-3-x86_64.cb, block size 131072.
[===============================================|] 1260/1260 100%
Exportable Squashfs 4.0 filesystem, lzma compressed, data block size 131072
       compressed data, compressed metadata, compressed fragments
       duplicates are removed
Filesystem size 8593.57 Kbytes (8.39 Mbytes)
       27.47% of uncompressed filesystem size (31283.16 Kbytes)
Inode table size 9782 bytes (9.55 Kbytes)
       22.16% of uncompressed inode table size (44141 bytes)
Directory table size 9679 bytes (9.45 Kbytes)
       35.74% of uncompressed directory table size (27085 bytes)
Number of duplicate files found 12
Number of inodes 1329
Number of files 1100
Number of fragments 81
Number of symbolic links  40
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 189
Number of ids (unique uids + gids) 2
Number of uids 1
       manutortosa (1000)
Number of gids 1
       users (100)
 
>> Cleaning work directories
:: libgnomecups DONE 
:: libgnomeprint DONE 
:: perl-xml-simple DONE 
:: icon-naming-utils DONE 
:: gnome-icon-theme DONE 
:: libgnomecanvas DONE 
:: libgnomeprintui DONE 
:: ccache DONE 
:: wxgtk DONE 
:: filezilla DONE 

>> Removing installed packages
:: Removing filezilla.
checking dependencies...
(1/1) removing filezilla                            [#####################################################] 100%
:: Removing wxgtk.
checking dependencies...
(1/1) removing wxgtk                                [#####################################################] 100%
:: Removing ccache.
checking dependencies...
(1/1) removing ccache                               [######################################################] 100%
:: Removing libgnomeprintui.
checking dependencies...
(1/1) removing libgnomeprintui                      [######################################################] 100%
:: Removing libgnomecanvas.
checking dependencies...
(1/1) removing libgnomecanvas                       [######################################################] 100%
:: Removing gnome-icon-theme.
checking dependencies...
(1/1) removing gnome-icon-theme                     [######################################################] 100%
:: Removing icon-naming-utils.
checking dependencies...
(1/1) removing icon-naming-utils                    [######################################################] 100%
:: Removing perl-xml-simple.
checking dependencies...
(1/1) removing perl-xml-simple                      [######################################################] 100%
:: Removing libgnomeprint.
checking dependencies...
(1/1) removing libgnomeprint                        [######################################################] 100%
:: Removing libgnomecups.
checking dependencies...
(1/1) removing libgnomecups                         [######################################################] 100%

:: Removing leftover files in _temp
real    5m40.515s
user    6m34.994s
sys     0m40.221s
>> All done

As you can see, the system automatically removes any package installed during the build. Now your bundle is done and your chroot is clean.

Chroot elements

`makebundle`

This is the main script of the Bundles Buildsystem, the script can compile the packages needed for a bundle, create the bundle using the compiled packages, clean the system or the whole pack, etc. Called without switches, bundle.sh do the whole process. Now lets see its switches:

-o: Example: ./makebundle filezilla -o; This switch ONLY creates the packages needed for the bundle and saves them into _temp, it searchs each package to build inside _temp and _pkgz, if the script finds a needed package, the package is installed and the build skipped, if the package is located in _pkgz and it's needed for the bundle itself it's copied to _temp.

-b: Example: ./makebundle filezilla -b; This switch ONLY creates the bundle, you need to have the packages needed for the bundle inside _temp, all the packages are backuped to _pkgz and it's recommended to clean the system with -c after the bundling.

-c: Example: ./makebundle filezilla -c; This switch ONLY clean the chroot environment, removing all the packages installed during the build, the src and pkg directories created inside each folder during the build as well as any build leftover file inside _temp.

Bundle components

Main folder

Each bundle needs a main folder with some special files on it, this example is the main folder of filezilla:

manutortosa@[chroot]:~/buildroot/bundles-testing/filezilla$ ls
ChangeLog  PKGBUILD  PKGINFO  filezilla.desktop  filezilla.install  filezilla.png

PKGINFO contains the main configuration to generate the bundle.
filezilla.desktop and filezilla.png are the desktop and icon related files to launch the bundle, defined in the PKGINFO.
PKGBUILD and Changelog are all the normal files you can find in any Chakra package.
You may need as well some patches or other related files, the version and release variables inside the PKGBUILD are used for naming and setting the version of the bundle, and its architecture is added automatically.

`PKGINFO`

This is the main configuration file to generate the bundle. It's a plain text file. Here it's the Filezilla one:

name=filezilla
desktop=filezilla.desktop
icon=filezilla.png
chroot=false
configure=external
binary=
pkgz=wxgtk filezilla
build=libgnomecups libgnomeprint perl-xml-simple icon-naming-utils gnome-icon-theme 
libgnomecanvas libgnomeprintui ccache wxgtk filezilla
env==
exec=/usr/bin/filezilla

PKGINFO
Find out information about how the PKGINFO file works.

You can find a template of the basic PKGINFO file in the chroot, PKGINFO.template, which you can use to create your own bundles. The final PKGINFO file will contain additional variables to the ones filled using PKGINFO.template as a template, which will be added by bundle.sh during the bundle creation, obtained from the PKGBUILD file and the bundles-i686-makepkg.conf or bundles-x86_64-makepkg.conf file, located in _buildsystem.

Navigation

SEARCH

TOOLBOX

LANGUAGES