Ticket #235 (closed defect: fixed)
CHOST is overwritten
| Reported by: | jkes | Owned by: | |
|---|---|---|---|
| Priority: | normal | Milestone: | |
| Component: | pkgcore | Version: | 0.5.9 |
| Keywords: | cross-compile | Cc: |
Description
Version 0.5.9 !!!
I have tried to cross-compile with pmerge.
The values CHOST, CBUILD or CTARGET, which are defined in a domain in a config file, are overwritten in ebd.py.
It works with following changes
=== modified file 'pkgcore/ebuild/ebd.py'
--- pkgcore/ebuild/ebd.py 2009-12-14 06:16:44 +0000
+++ pkgcore/ebuild/ebd.py 2010-01-20 12:35:29 +0000
@@ -97,7 +97,7 @@
for x in ('chost', 'cbuild', 'ctarget'):
val = getattr(pkg, x)
if val is not None:
- self.env[x.upper()] = val
+ self.env.setdefault(x.upper(),val)
if "PYTHONPATH" in os.environ:
self.env["PYTHONPATH"] = os.environ["PYTHONPATH"]
Now CHOST, CBUILD or CTARGET are only overwritten, when there are not defined.
Change History
comment:2 Changed 7 months ago by jkes
The config file pkgcore_arm_1.conf contains all the definitions I am using.
Create some files and folders in a new and empty folder /test4
mkdir -p /test4/packages mkdir -p /test4/var/db/pkg mkdir -p /test4/var/lib/portage touch /test4/var/lib/portage/world
start the debugger with
pmerge -dv wget
set a breakpoint in line 99 in pkgcore/ebuild/ebd.py
for x in ('chost', 'cbuild', 'ctarget'):
val = getattr(pkg, x)
if val is not None: # <<< set breakpoint here
self.env.setdefault(x.upper(),val)
When the breakpoint is hit for the first time the values of CHOST, CBUILD and CTARGET are:
self.env.CHOST: i686-pc-linux-gnu
self.env.CBUILD: arm-unknown-linux-gnu
self.env.CTARGET: not defined
By stepping through the code with the debugger I get the values of variable val for
chost: arm-unknown-linux-gnu
cbuild: arm-unknown-linux-gnu
ctarget: arm-unknown-linux-gnu
When I change the name in 'livefs domain' to
CHOST 'dummy1'
the values of chost, cbuild and ctarget change also to 'dummy1'.
chost, cbuild and ctarget get their value only from CHOST in 'livefs domain' in pkgcore_arm_1.conf
My workaround for this is
self.env.setdefault(x.upper(),val)
in line 100 in ebd.py.
I am using the following /etc/pkgcore.conf file
[autoload-dhcpformat] class=pkgcore.config.basics.parse_config_file parser=pkgcore.config.dhcpformat.config_from_file path=/etc/pkgcore_arm_1.conf
comment:3 Changed 7 months ago by jkes
The following config file pkgcore_arm_1.conf contains all the definitions I am using.
# Definition for an arm cross-compiler
#
# root and config root are /test4
#
# I have created this file by using:
# export CONFIG_ROOT=/test4
# pconfig dump
# on an existing file /test4/etc/make.conf.
# I have been able to use /test4/etc/make.conf with emerge-wrapper.
#
# The lines in the output of 'pconfig dump' which I had to change
# afterwards to get it working are marked with ERROR
#
# When I use this file I don't use ROOT or CONFIG_ROOT anymore
#
'/usr/portage' {
# typename of this section: repo
class pkgcore.ebuild.repository.UnconfiguredTree;
# type: refs:cache
cache 'portdir cache';
# type: list
default_mirrors 'ftp://ftp.gentoo.mesh-solutions.com/gentoo//distfiles';
# type: ref:eclass_cache
# eclass_cache 'eclass stack'; # ERROR: alias doesn't work here
eclass_cache '/usr/portage/eclass';
# type: bool
ignore_paludis_versioning False;
# type: str
location '/usr/portage';
# type: lazy_ref:syncer
sync '/usr/portage syncer';
}
'/usr/portage syncer' {
# typename of this section: syncer
class pkgcore.sync.rsync.rsync_timestamp_syncer;
# type: str
basedir '/usr/portage';
# type: str
retries '3';
# type: str
uri 'rsync://rsync.gentoo.org/gentoo-portage';
}
'/usr/portage/eclass' {
# typename of this section: eclass_cache
class pkgcore.ebuild.eclass_cache.cache;
# type: str
path '/usr/portage/eclass';
# type: str
portdir '/usr/portage';
}
#'autoload-portage' {
# # typename of this section: configsection
# class pkgcore.ebuild.portage_conf.config_from_make_conf;
#}
'basic-formatter' {
# typename of this section: pmerge_formatter
class pkgcore.ebuild.formatter.factory;
}
'binary_debug_trigger' {
# typename of this section: trigger
class pkgcore.merge.triggers.BinaryDebug;
# type: str
mode 'strip';
}
'binpkg' {
# typename of this section: repo
class pkgcore.binpkg.repository.tree;
# type: bool
ignore_paludis_versioning False;
# type: str
location '/test4/packages';
}
'buildpkg_trigger' {
# typename of this section: trigger
class pkgcore.merge.triggers.SavePkg;
# type: bool
pristine False;
# type: ref:repo
target_repo 'binpkg';
}
#'eclass stack' {
# # typename of this section: eclass_cache
# class pkgcore.config.basics.section_alias;
# typename 'eclass_cache'; # ERROR: line was missing
# # type: ref:eclass_cache
# target '/usr/portage/eclass';
#}
#'eclass stack' {
# # typename of this section: eclass_cache
# # class pkgcore.ebuild.eclass_cache
# class pkgcore.config.basics.section_alias;
# typename 'eclass_cache'; # ERROR: line was missing
# # type: ref:eclass_cache
# target '/usr/portage/eclass';
#}
'fetcher' {
# typename of this section: fetcher
class pkgcore.fetch.custom.fetcher;
# type: str
ACCEPT_KEYWORDS 'arm';
# type: str
ARCH 'arm';
# type: str
AUTOCLEAN 'yes';
# type: str
CBUILD 'i686-pc-linux-gnu';
# type: str
CFLAGS '-Os -pipe -mcpu=arm1136jf-s -fomit-frame-pointer'; # -I/usr/arm-unknown-linux-gnu/usr/include/ -I/usr/arm-unknown-linux-gnu/include/;
# type: str
CHOST 'arm-unknown-linux-gnu';
# type: str
CLEAN_DELAY '5';
# type: str
COLLISION_IGNORE '/lib/modules';
# type: str
CONFIG_PROTECT '/etc';
# type: str
CONFIG_PROTECT_MASK '/etc/env.d';
# type: str
CXXFLAGS '-Os -pipe -mcpu=arm1136jf-s -fomit-frame-pointer'; # -I/usr/arm-unknown-linux-gnu/usr/include/ -I/usr/arm-unknown-linux-gnu/include/';
# type: str
ELIBC 'glibc';
# type: str
EMERGE_WARNING_DELAY '10';
# type: str
E_MACHINE 'EM_ARM';
# type: str
FEATURES '-collision-protect sandbox buildpkg noman noinfo nodoc';
# type: str
HOSTCC 'i686-pc-linux-gnu-gcc';
# type: str
INPUT_DEVICES 'evdev keyboard mouse touchscreen';
# type: str
# LDFLAGS ''; #'-L/usr/arm-unknown-linux-gnu/lib -L/usr/arm-unknown-linux-gnu/usr/lib';
# type: str
LIBDIR_amd64 'lib64';
# type: str
LIBDIR_arm 'lib';
# type: str
MAKEOPTS '-j1';
# type: str
MARCH_TUNE '-mcpu=arm1136jf-s';
# type: str
PKG_CONFIG_PATH '/test4/usr/lib/pkgconfig/';
# type: str
PORTAGE_BINHOST_CHUNKSIZE '3000';
# type: str
PORTAGE_COMPRESS_EXCLUDE_SUFFIXES 'css gif htm[l]? jp[e]?g js pdf png';
# type: str
PORTAGE_ELOG_CLASSES 'log warn error';
# type: str
PORTAGE_ELOG_MAILFROM 'portage@localhost';
# type: str
PORTAGE_ELOG_MAILSUBJECT '[portage] ebuild log for ${PACKAGE} on ${HOST}';
# type: str
PORTAGE_ELOG_MAILURI 'root';
# type: str
PORTAGE_ELOG_SYSTEM 'save_summary echo';
# type: str
PORTAGE_FETCH_CHECKSUM_TRY_MIRRORS '5';
# type: str
PORTAGE_FETCH_RESUME_MIN_SIZE '350K';
# type: str
PORTAGE_INST_GID '1000'; # some user
# type: str
PORTAGE_INST_UID '1000'; # some user
# type: str
PORTAGE_RSYNC_OPTS '--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages';
# type: str
PORTAGE_TMPDIR '/test4/tmp/';
# type: str
PORTAGE_WORKDIR_MODE '0700';
# type: str
ROOT '/test4/';
# type: str
RPMDIR '/usr/portage/rpm';
# type: str
UCLIBC_CPU_DEFAULT 'GENERIC_ARM';
# type: str
USE 'arm zlib bindist make-symlinks minimal input_devices_keyboard input_devices_evdev video_cards_fbdev video_cards_dummy';
# type: str
USE_EXPAND 'video_cards input_devices';
# type: str
USE_ORDER 'env:pkg:conf:defaults:pkginternal:env.d';
# type: str
VIDEO_CARDS 'fbdev dummy';
# type: str
command '/usr/bin/wget -t 5 -T 60 --passive-ftp -O "${DISTDIR}/${FILE}" "${URI}"';
# type: str
distdir '/usr/portage/distfiles';
# type: str
resume_command '/usr/bin/wget -c -t 5 -T 60 --passive-ftp -O "${DISTDIR}/${FILE}" "${URI}"';
}
'glsa' {
# typename of this section: pkgset
class pkgcore.config.basics.section_alias;
typename 'pkgset'; # ERROR line was missing
# type: ref:pkgset
target 'vuln';
}
'installed' {
# typename of this section: pkgset
class pkgcore.pkgsets.installed.Installed;
# type: refs:repo
vdb 'vdb';
}
'livefs domain' {
# typename of this section: domain
class pkgcore.ebuild.domain.domain;
default true;
# type: list
ACCEPT_KEYWORDS 'arm';
# type: str
ARCH 'arm';
# type: str
AUTOCLEAN 'yes';
# type: str
CBUILD 'i686-pc-linux-gnu';
# type: str
CFLAGS '-Os -pipe -mcpu=arm1136jf-s -fomit-frame-pointer'; #'-I/usr/arm-unknown-linux-gnu/usr/include/ -I/usr/arm-unknown-linux-gnu/include/';
# type: str
CHOST 'arm-unknown-linux-gnu';
# type: str
CLEAN_DELAY '5';
# type: str
COLLISION_IGNORE '/lib/modules';
# type: list
CONFIG_PROTECT '/etc';
# type: list
CONFIG_PROTECT_MASK '/etc/env.d';
# type: str
CXXFLAGS '-Os -pipe -mcpu=arm1136jf-s -fomit-frame-pointer'; # -I/usr/arm-unknown-linux-gnu/usr/include/ -I/usr/arm-unknown-linux-gnu/include/';
# type: str
ELIBC 'glibc';
# type: str
EMERGE_WARNING_DELAY '10';
# type: str
E_MACHINE 'EM_ARM';
# type: list
FEATURES '-collision-protect' 'sandbox' 'buildpkg' 'noman' 'noinfo' 'nodoc';
# type: str
HOSTCC 'i686-pc-linux-gnu-gcc';
# type: str
INPUT_DEVICES 'evdev keyboard mouse touchscreen';
# type: str
# LDFLAGS ''; #'-L/usr/arm-unknown-linux-gnu/lib -L/usr/arm-unknown-linux-gnu/usr/lib';
# type: str
LIBDIR_amd64 'lib64';
# type: str
LIBDIR_arm 'lib';
# type: str
MAKEOPTS '-j1';
# type: str
MARCH_TUNE '-mcpu=arm1136jf-s';
# type: str
PKG_CONFIG_PATH '/test4/usr/lib/pkgconfig/';
# type: str
PORTAGE_BINHOST_CHUNKSIZE '3000';
# type: str
PORTAGE_COMPRESS_EXCLUDE_SUFFIXES 'css gif htm[l]? jp[e]?g js pdf png';
# type: str
PORTAGE_ELOG_CLASSES 'log warn error';
# type: str
PORTAGE_ELOG_MAILFROM 'portage@localhost';
# type: str
PORTAGE_ELOG_MAILSUBJECT '[portage] ebuild log for ${PACKAGE} on ${HOST}';
# type: str
PORTAGE_ELOG_MAILURI 'root';
# type: str
PORTAGE_ELOG_SYSTEM 'save_summary echo';
# type: str
PORTAGE_FETCH_CHECKSUM_TRY_MIRRORS '5';
# type: str
PORTAGE_FETCH_RESUME_MIN_SIZE '350K';
# type: str
PORTAGE_INST_GID '1000'; # some user
# type: str
PORTAGE_INST_UID '1000'; # some user
# type: str
PORTAGE_RSYNC_OPTS '--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages';
# type: str
PORTAGE_TMPDIR '/test4/tmp/';
# type: str
PORTAGE_WORKDIR_MODE '0700';
# type: str
ROOT '/test4/';
# type: str
RPMDIR '/usr/portage/rpm';
# type: str
UCLIBC_CPU_DEFAULT 'GENERIC_ARM';
# type: list
USE 'arm' 'zlib' 'bindist' 'make-symlinks' 'minimal' 'input_devices_keyboard' 'input_devices_evdev' 'video_cards_fbdev' 'video_cards_dummy';
# type: str
USE_EXPAND 'video_cards input_devices';
# type: str
USE_ORDER 'env:pkg:conf:defaults:pkginternal:env.d';
# type: str
VIDEO_CARDS 'fbdev dummy';
# type: list
bashrc '/test4/etc/portage/bashrc';
# type: ref:fetcher
fetcher 'fetcher';
# type: str
name 'livefs domain';
# type: ref:profile
profile 'profile';
# type: lazy_refs:repo
# repositories 'repo-stack' 'binpkg'; # ERROR: alias doesn't work here
repositories '/usr/portage' 'binpkg';
# type: str
root '/test4/';
# type: lazy_refs:trigger
triggers 'buildpkg_trigger' 'binary_debug_trigger';
# type: lazy_refs:repo
vdb 'vdb';
}
'paludis-formatter' {
# typename of this section: pmerge_formatter
class pkgcore.ebuild.formatter.factory;
}
'pkgcore-formatter' {
# typename of this section: pmerge_formatter
class pkgcore.ebuild.formatter.factory;
}
'portage-formatter' {
# typename of this section: pmerge_formatter
class pkgcore.ebuild.formatter.factory;
default true;
}
'portage-verbose-formatter' {
# typename of this section: pmerge_formatter
class pkgcore.ebuild.formatter.portage_verbose_factory;
}
'portdir' {
# typename of this section: repo
class pkgcore.config.basics.section_alias;
typename 'repo'; # ERROR: line was missing
# type: ref:repo
target '/usr/portage';
}
'portdir cache' {
# typename of this section: cache
class pkgcore.cache.metadata.database;
# type: ref:eclass_cache
eclasses '/usr/portage/eclass';
# type: str
location '/usr/portage';
# type: bool
readonly True;
}
'profile' {
# typename of this section: profile
class pkgcore.ebuild.profiles.OnDiskProfile;
# type: str
basepath '/usr/portage/profiles';
# type: str
profile 'embedded';
}
'repo-stack' {
# typename of this section: repo
class pkgcore.config.basics.section_alias;
typename 'repo'; # ERROR: line was missing
# type: ref:repo
target '/usr/portage';
}
'system' {
# typename of this section: pkgset
class pkgcore.pkgsets.system.SystemSet;
# type: ref:profile
profile 'profile';
}
'vdb' {
# typename of this section: repo
class pkgcore.vdb.ondisk.tree;
# type: str
cache_location '/test4/var/cache/edb/dep/var/db/pkg';
# type: str
location '/test4/var/db/pkg';
}
'versioned-installed' {
# typename of this section: pkgset
class pkgcore.pkgsets.installed.VersionedInstalled;
# type: refs:repo
vdb 'vdb';
}
'vuln' {
# typename of this section: pkgset
class pkgcore.ebuild.portage_conf.SecurityUpgradesViaProfile;
# type: ref:repo
# ebuild_repo 'repo-stack'; # ERROR: alias doesn't work here
ebuild_repo '/usr/portage';
# type: ref:profile
profile 'profile';
# type: ref:repo
vdb 'vdb';
}
'world' {
# typename of this section: pkgset
class pkgcore.pkgsets.filelist.WorldFile;
# type: str
location '/test4/var/lib/portage/world';
#location '/var/lib/portage/world'; ## ERROR: config_root is missing here
}
comment:4 Changed 7 months ago by ferringb
- Status changed from new to closed
- Resolution set to fixed
Try 0.5.10 please; the problem I have w/ the fix you're using is that for binpkgs it would override the cbuild/ctarget/chost w/ the current domains settings- this isn't always correct (the pkg was built w/ those settings, that's what they are).
What *was* screwed up however was the fallback setting of CBUILD/CTARGET to CHOST- it was automatically stomping the values- specifically look for scope_update and cbuild/ctarget in pkgcore/ebuild/repository.py
The fix was basically-
scope_update = {'chost': chost}
-scope_update.update((x, domain_settings.get(x.upper, chost))
+scope_update.update((x, domain_settings.get(x.upper(), chost))
for x in ('cbuild', 'ctarget'))
Since it was trying to look up the method rather then the method results (uppercase varname), it was just stomping cbuild/ctarget incorrectly.
This should restore CC support- if not, let me know, I'm expecting a 0.5.11 release in the next week or so.
Note: See
TracTickets for help on using
tickets.
I'm not convinced this is proper- how are you seeing this exactly?
Basically, the pkgs configuration wrapping (or in the case of binpkgs, hardcoded chost/cbuild/ctarget) ought to be valid- those settings work their way down from the domain after all.
Off the top of my head, I don't quite see how it's possible to get a situation where domain settings (where those env values come from) differ from the pkg- exempting binpkgs, which the values shouldn't change for.
Comments?