15:04 <slangasek> #startmeeting 15:04 <meetingology> Meeting started Wed Sep 11 15:04:18 2013 UTC. The chair is slangasek. Information about MeetBot at http://wiki.ubuntu.com/meetingology. 15:04 <meetingology> 15:04 <meetingology> Available commands: #accept #accepted #action #agree #agreed #chair #commands #endmeeting #endvote #halp #help #idea #info #link #lurk #meetingname #meetingtopic #nick #progress #rejected #replay #restrictlogs #save #startmeeting #subtopic #topic #unchair #undo #unlurk #vote #voters #votesrequired 15:04 <slangasek> [TOPIC] lightning round 15:04 <slangasek> $ echo $(shuf -e barry doko stgraber jodh ev bdmurray slangasek cjwatson xnox stokachu) 15:04 <slangasek> ev xnox bdmurray doko slangasek cjwatson stgraber barry jodh stokachu 15:04 <slangasek> ev: still in other meetings? 15:04 <ev> ja 15:04 <xnox> me? 15:04 <ev> (upstream merger training) 15:04 <slangasek> ev: do you have a report to dump us, or should we skip you for now? 15:05 <ev> please skip 15:05 <slangasek> ok 15:05 <slangasek> xnox: 15:05 <xnox> - cross-compile qml extensions using cmake, turned out to be much 15:05 <xnox> easier than qmake. Posted to ubuntu-phone mailing list and a wiki 15:05 <xnox> page https://wiki.ubuntu.com/Touch/CrossCompile 15:05 <xnox> - uploaded ubiquity with a few merge proposals merged & a split of 15:05 <xnox> ubuntuone plugin. Thus it should be possible to seed/unseed 15:05 <xnox> ubuntuone plugin page on per-image basis. 15:05 <xnox> - ongoing work to fix bug #1216043 15:05 <xnox> - updating kernel-config of linux-goldfish, at the moment it's not 15:05 <xnox> sufficient to neither boot ubuntu nor stock android images. Tracking 15:05 <ubottu> bug 1216043 in hw-detect (Ubuntu Saucy) "driver-inject-disk: in target debs are installed before kernel has been" [High,Confirmed] https://launchpad.net/bugs/1216043 15:05 <xnox> bug #1222772 15:05 <ubottu> bug 1222772 in linux-goldfish (Ubuntu) "Config changes request to support Ubuntu Touch & stock Android" [Undecided,New] https://launchpad.net/bugs/1222772 15:05 <xnox> - prepare/review/submit upstart slides for linuxcon/plumbers 15:05 <xnox> - short week: piloting on the 5th, vacation on the 10th (submitted 15:05 <xnox> application to naturalise as British) 15:05 <xnox> .. 15:05 <bdmurray> SRU verification of bug 1211511 15:05 <bdmurray> SRU verification of Q, R ubuntu-release-upgrader bugs 15:05 <bdmurray> updated ubuntu-release-upgrader DevelReleaseAnnouncement to say BETA 15:05 <bdmurray> tested and uploaded cjwatson's apt-clone fix for bug 1212025 15:05 <ubottu> bug 1211511 in update-manager (Ubuntu Raring) "update-manager hides but wants to install ignored phased updates" [High,Fix released] https://launchpad.net/bugs/1211511 15:05 <ubottu> bug 1212025 in ubuntu-release-upgrader (Ubuntu) "Upgrading to Saucy crashes with a unicode error" [Undecided,Confirmed] https://launchpad.net/bugs/1212025 15:06 <cjwatson> ta 15:06 <bdmurray> uploaded fix for ubuntu-release-upgrader bug 1221307 15:06 <bdmurray> research into verification of the fix for bug 1122596 and bug 1058884 15:06 <bdmurray> research into bug 1220898 and irc discussion regarding it 15:06 <ubottu> bug 1221307 in ubuntu-release-upgrader (Ubuntu) "SystemError message is unhelpful and causes a crash for non-English languages" [Medium,Fix released] https://launchpad.net/bugs/1221307 15:06 <ubottu> bug 1122596 in libappindicator (Ubuntu Precise) "Race condition in app_indicator_init() causes application crash" [High,Fix committed] https://launchpad.net/bugs/1122596 15:06 <ubottu> bug 1058884 in python3.3 (Ubuntu Raring) "Race condition in py_compile corrupts pyc files" [High,Fix committed] https://launchpad.net/bugs/1058884 15:06 <ubottu> bug 1220898 in ubuntu-release-upgrader (Ubuntu) "generic kernel gets installed in ubuntustudio upgrade." [High,Triaged] https://launchpad.net/bugs/1220898 15:06 <bdmurray> code review of slangasek's ubuntu-release-upgrader cruft removal merge proposal 15:06 <bdmurray> fixed an issue with arsenal reports regarding multiple teams 15:06 <bdmurray> tested multiple kernels for bug 1218004 15:06 <ubottu> bug 1218004 in linux (Ubuntu) "Apple Wireless Trackpad causes kernel oops" [High,Confirmed] https://launchpad.net/bugs/1218004 15:06 <bdmurray> ⌁ done 15:09 <doko> - had vacation days Friday and this week, done some tax stuff, nothing work related 15:09 <doko> (done) 15:11 <slangasek> * SecureBoot: 15:11 <slangasek> * fixed sbsigntool to be able to actually reproduce Microsoft-signed binaries byte-for byte, letting us automatically validate the results of SysDev signing 15:11 <slangasek> * uploaded shim-signed, which fixes various bugs and brings in fun new features like netboot support 15:11 <slangasek> * worked with cjwatson to get a pregenerated UEFI netboot image for grub2 - so now we have full SecureBoot netboot support in saucy 15:11 <slangasek> * now need to look at getting this all backported to precise 15:11 <slangasek> * updated mountall for various bugfixes (bug #503003, bug #731800); also makes mountall debuggable without having to edit the upstart job; now dealing with a regression, bug #1223745. 15:11 <ubottu> bug 503003 in mountall (Ubuntu) "multiple entries in fstab with same mount-point" [Medium,Fix released] https://launchpad.net/bugs/503003 15:11 <slangasek> * TODO: 15:11 <ubottu> bug 731800 in mountall (Ubuntu) "mountall assert failure: mountall.c:1411: Assertion failed in skip_mount: (! mnt->mounted) || needs_remount (mnt)" [Medium,Fix released] https://launchpad.net/bugs/731800 15:11 <slangasek> * parted still not done 15:11 <ubottu> bug 1223745 in mountall (Ubuntu) "Dependency resolver causes hang on boot" [Undecided,New] https://launchpad.net/bugs/1223745 15:11 <slangasek> * plumbers next week 15:11 <slangasek> * booking travel for the client sprint at the end of October (you guys don't have to go, except cjwatson :) 15:11 <slangasek> * get air conditioning, since August has come again 15:11 <slangasek> (done) 15:11 <cjwatson> Click: 15:11 <cjwatson> Finished deploying preinstalled app handling, although there's a glitch involving needing to re-run system hooks. 15:11 <cjwatson> Wrote a click(1) manual page. 15:11 <cjwatson> Implemented removal support. (Harder than it looked due to the necessary garbage collection semantics.) 15:11 <cjwatson> Implemented search by name/details in the PackageKit plugin. 15:12 <cjwatson> "click list --manifest" tweak to speed up the system settings UI (bug 1221760). 15:12 <ubottu> bug 1221760 in click (Ubuntu) "Having "pkgdir" in the manifest info would be useful" [High,Fix released] https://launchpad.net/bugs/1221760 15:12 <cjwatson> Various bug fixes (bug 1197047, bug 1215480, bug 1218483). 15:12 <cjwatson> OEM bugs: 15:12 <ubottu> bug 1197047 in upstart-app-launch (Ubuntu Saucy) "SDK applications create /tmp/*.sci files" [High,In progress] https://launchpad.net/bugs/1197047 15:12 <cjwatson> Fixed bug 1097570, pending testing that I haven't broken ordinary booting from hard disk before releasing to saucy. 15:12 <ubottu> bug 1215480 in click (Ubuntu) "wrong permissions of click packages due to adb shell umask of 0000" [High,Fix released] https://launchpad.net/bugs/1215480 15:12 <ubottu> bug 1218483 in click (Ubuntu) "Installation errors are not reported" [High,Fix released] https://launchpad.net/bugs/1218483 15:12 <cjwatson> Attempted to track down bug 1215458, but stymied by parted's debug symbols being unavailable. Fixed the Debian packaging for that and just received the results of another test run. 15:12 <ubottu> bug 1097570 in grub2 (Ubuntu Raring) "grub2-signed can not find the right device when there are two filesystems containing the file '.disk/info'." [High,Triaged] https://launchpad.net/bugs/1097570 15:12 <ubottu> bug 1215458 in partman-basicfilesystems (Ubuntu) "Auto-partitioning with LVM on Advanced Format Disks causes parted_server segfault" [High,In progress] https://launchpad.net/bugs/1215458 15:12 <cjwatson> Investigated bug 1197766 and posted a candidate patch. 15:12 <ubottu> bug 1197766 in OEM Priority Project "Different partition layout after recovery with keep home partition" [Critical,Confirmed] https://launchpad.net/bugs/1197766 15:12 <cjwatson> Misc: 15:12 <cjwatson> Rebootstrapped gradle. 15:12 <cjwatson> Various minor transitions, merges, etc.; mostly trying to reduce the list of outstanding changes in -proposed. 15:12 <cjwatson> To do: 15:12 <cjwatson> Continue to analyse crash data from 1215458. 15:12 <cjwatson> Organise travel for next client sprint. 15:12 <cjwatson> Start on chroot-management code in click to improve architecture-specific handling. 15:12 <cjwatson> .. 15:13 <stgraber> Blueprint-related work: 15:13 <stgraber> - Image based updates (BLUEPRINT: foundations-1305-image-based-updates) 15:13 <stgraber> - Made system-images the default 15:13 <stgraber> - Worked with QA to tweak the channel setup and help fix some tests 15:13 <stgraber> - Re-designed channels.json to support channel aliases and hidden channels 15:13 <stgraber> - Did some refactoring in the server code, moved a bunch of stuff from independent scripts to the library and now covered by tests (fixed a few bugs in the process) 15:13 <stgraber> - Implemented and tested support for channel aliases (daily => saucy, stable => saucy) 15:13 <stgraber> - Started working on the import-cdimage rewrite to be much more flexible, drive as many channels as needed, do image expiry, auto-cleanup, shared image pool, support watching http urls (for customization tarballs). 15:13 <stgraber> 15:13 <stgraber> Other work: 15:13 <stgraber> - LXC 15:13 <stgraber> - Linux Plumbers mini-conference preparation 15:13 <stgraber> - Some code review 15:13 <stgraber> - Started working on a new website and new domain name for the project 15:13 <stgraber> - Spent a couple of hours updating debian/copyright after the upstream licensing changes (it used to be completely wrong...) 15:13 <stgraber> - Trying to get LXC to behave on an old 2.6.32 based Android phone so I can give my talk next week... 15:13 <stgraber> - Generate the 1.0~alpha1 tarballs, getting a couple of fixes cherry-picked to fix regressions on Ubuntu Touch for an upload tonight or tomorrow morning. 15:13 <stgraber> - Other 15:13 <stgraber> - Various additions to writable-paths in Ubuntu Touch 15:13 <stgraber> - Some rework of the initrd hook to fix file/directory ownership for writable paths. 15:13 <stgraber> - Code reviews and tests for the guys working on Ubuntu Touch customizations 15:13 <stgraber> - Debugged sssd 1.11 breaking with samba4, now fixed in upstream git and cherry-picked to saucy. 15:13 <stgraber> 15:13 <stgraber> TODO (hopefully this week): 15:13 <stgraber> - Upload LXC 1.0~alpha1 (waiting for a fix from Dwight) 15:13 <stgraber> - Finish the import-cdimage rewrite (now scheduled to land on Monday) 15:14 <stgraber> - Implement the boot-time hooks for Ubuntu touch 15:14 <stgraber> - Plumbers 2013 prep 15:14 <stgraber> (DONE) 15:14 <barry> system-image: LP: #1215959; LP: #1221841; LP: #1196991 (ongoing). 1.5 and 1.5.1 uploaded. weekly meeting. 15:14 <ubottu> Launchpad bug 1215959 in Ubuntu system image "Report image versions" [High,Fix committed] https://launchpad.net/bugs/1215959 15:14 <ubottu> Launchpad bug 1221841 in Ubuntu system image "Support the new channels.json format in a backward compatible way" [Critical,Fix committed] https://launchpad.net/bugs/1221841 15:14 <ubottu> Launchpad bug 1196991 in Ubuntu system image "Support the new download dbus service" [High,In progress] https://launchpad.net/bugs/1196991 15:14 <barry> other: tox issue #122 (affected system-image local testing). virtualenv, pip, tox, nose2 packaging work. dmb meeting. awaiting FFes. LP: #1223004. LP: #1223001. LP: #1223483. Debian bug #722295. pip 1.4.1-2. Debugging kernel regressions LP: #1223352 and LP: #1223408. 15:14 <ubottu> Launchpad bug 1223004 in python-oauthlib (Ubuntu) "[FFe] sync with debian which has 0.5.1" [Wishlist,Fix released] https://launchpad.net/bugs/1223004 15:14 <ubottu> Launchpad bug 1223001 in python-pip (Ubuntu) "[FFe] sync python-pip 1.4.1 from debian" [Undecided,New] https://launchpad.net/bugs/1223001 15:14 <ubottu> Launchpad bug 1223483 in python-virtualenv (Ubuntu) "[FFe] sync 1.10.1 from Debian" [Undecided,New] https://launchpad.net/bugs/1223483 15:14 <ubottu> Debian bug 722295 in python-pip "python-pip: ca-certificates needs to be Depends rather than Recommends" [Normal,Fixed] http://bugs.debian.org/722295 15:14 <barry> ↺ 15:14 <jodh> * upstart: 15:14 <jodh> - Fixed bug in Upstart apport hook. 15:14 <jodh> - Fixed bug 1221466 and bug 1222702. Awaiting feedback on MP with new 15:14 <jodh> tests: 15:15 <ubottu> bug 1221466 in upstart (Ubuntu) "upstart-file-bridge 'EVENT=create' not working for directories" [Undecided,In progress] https://launchpad.net/bugs/1221466 15:15 <jodh> https://code.launchpad.net/~jamesodhunt/upstart/bug-1221466+1222702/+merge/184829 15:15 <ubottu> bug 1222702 in upstart (Ubuntu) "upstart-file-bridge assert failure: upstart-file-bridge.c:1162: Assertion failed in delete_handler: strstr (path, dir->path) == path" [Medium,In progress] https://launchpad.net/bugs/1222702 15:15 <jodh> - Fixed failing quiesce tests. Awaiting feedback on updated MP: 15:15 <jodh> https://code.launchpad.net/~jamesodhunt/upstart/test-quiesce-cleanup/+merge/182698 15:15 <jodh> - Investigating bug 1222705. 15:15 <ubottu> bug 1222705 in upstart (Ubuntu) "init assert failure: alloc.c:633: Assertion failed in nih_unref: ref != NULL" [Medium,New] https://launchpad.net/bugs/1222705 15:15 <jodh> - Investigating suspected race in system init re-exec test causing 15:15 <jodh> occasional failure. 15:15 <jodh> 15:15 <jodh> - Updated Technical Overview for Upstart: 15:15 <jodh> https://wiki.ubuntu.com/SaucySalamander/TechnicalOverview#Upstart_1.10 15:15 <jodh> * misc: 15:15 <jodh> - Linux Plumbers preparation. 15:15 <jodh> 奇 15:15 <xnox> jodh: re upstart-file-bridge MP: looks good, but didn't run the tests yet, want to run them before merging. 15:16 <stokachu> bug 1160490 - i think slangasek tasked himself to upload? and having a really tough time getting traction on bug 1157943 15:16 <jodh> xnox: great, thanks. 15:16 <ubottu> bug 1160490 in ifupdown (Ubuntu Raring) "race condition updating statefile" [Medium,In progress] https://launchpad.net/bugs/1160490 15:16 <ubottu> bug 1157943 in apt (Ubuntu Precise) "apt-get update fails hash checks on https repositories when file size changes" [Undecided,New] https://launchpad.net/bugs/1157943 15:16 <stokachu> done 15:16 <xnox> slangasek: i guess the test-quiesce-cleanup branch is for you to further comment on =) 15:17 <slangasek> barry: do you have an eta for bug #1196991? that seems to be the critical piece we're still missing 15:17 <ubottu> bug 1196991 in Ubuntu system image "Support the new download dbus service" [High,In progress] https://launchpad.net/bugs/1196991 15:17 <slangasek> stokachu: yeah, 1160490 still on my radar 15:17 <stokachu> slangasek: thanks man, arges ^ 15:18 <arges> slangasek: thanks 15:18 <barry> slangasek: if nothing else intrudes, from a few days to eow. 15:18 <barry> (some refactoring is required, but this will also close a few other bugs) 15:18 <slangasek> xnox, jodh: test-quiesce-cleanup> right, I need to circle around on that one; I still have my doubts that these changes are correct, but need to actually review the code to be sure 15:19 <slangasek> barry: ok, perfect! do you need any help keeping out intruders? I can post sentries ;) 15:20 <barry> slangasek: i might! ;) 15:21 <stgraber> slangasek: I think I've been the biggest source of delays to barry's work so far but that's because of other bits that were higher priority. I don't think I've got anything else to get in before the D/L service work now, so hopefully barry can focus on that :) 15:21 <slangasek> stokachu: 1157943> if you want a response from David, you're going to need to reach out to him more directly than a "ping" on the bug log 15:21 <barry> yay! :) 15:21 <stokachu> slangasek: so just start emailing him directly then? 15:21 <stokachu> slangasek: didnt want to piss him off 15:22 <slangasek> stgraber, barry: right, sounds good 15:22 <slangasek> stokachu: by "more directly", I mean you might want to at least make it clear in your email to the bug that you're addressing him :) 15:22 <stokachu> slangasek: ok 15:24 <slangasek> stokachu: I always think it's best to keep the discussion on the public bug tracker, but use his name / make it clear you're looking for his input. Remember he's a non-Canonical upstream, so he also doesn't have any obligation to help us on your time line 15:26 <slangasek> any other questions / discussion over status? 15:27 <stokachu> gotcha 15:28 <slangasek> stokachu: oh, and if you want to get him on IRC, he's DonKult and hangs around in some ubuntu channels 15:28 <slangasek> (sometimes) 15:28 <stokachu> slangasek: ok ill keep an eye out for him, looks like he's not on any channels atm 15:29 <slangasek> [TOPIC] AOB 15:29 <slangasek> anything else on people's minds? 15:30 <slangasek> [TOPIC] LXC 15:30 <slangasek> new topic, then :) 15:30 <slangasek> you are now a captive audience for stgraber to practice his LXC material for Plumbers next week ;) 15:32 <stgraber> Alright, so LXC. 15:32 <stgraber> I guess everyone knows what LXC is and that most of you already use it voluntarily or not. 15:32 <stgraber> The LXC project is about writing convenient tools, libraries and bindings to the various containment APIs available in the kernel. 15:32 <stgraber> As it's today, that's: namespaces (ipc, pid, utc, mount, network and user), cgroups (all the controllers are supported), capabilities, apparmor (custom profiles per container) and seccomp (custom profile per container). 15:32 <stgraber> 15:32 <stgraber> The current stable version of LXC is 0.9, released back in April after a 5 months development cycle. 15:32 <stgraber> The next release is going to be 1.0, for that one we've agreed on a much longer development cycle which started back in April with the release planned in early February (10 months). 15:32 <stgraber> (I've got a bunch of stuff ready to paste but will do so in batches, so don't hesitate to ask questions at any time and I'll stop) 15:33 <stgraber> The longer development cycle is explained by a lot of work needing to happen and some high expectations for a 1.0 release. 15:33 <stgraber> By the time we release 1.0, we're planning on doing: 15:33 <stgraber> - rewrite all our tools to use our public library instead of internal functions 15:33 <stgraber> - release a stable version of the liblxc API 15:33 <stgraber> - release a stable API for all supported bindings (python3, lua and go) 15:33 <stgraber> - version our internal management protocol and have limited backward compatibility there 15:33 <stgraber> - get fully functional user namespace support 15:33 <stgraber> - support kernels from 2.6.32 kernel to whatever is latest at the time of release 15:33 <stgraber> - support for building under the Android NDK 15:33 <stgraber> - support for container cloning, snapshotting and backing store plugins 15:33 <stgraber> - and a LOT of refactoring, bug fixes and templates improvement 15:33 <stgraber> - actively maintain the 1.0 branch for a minimum of 2 years 15:34 <stgraber> LXC is also in a transitional phase at the moment where Daniel Lezcano is still the only commiter to our master branch and Serge Hallyn and I are the only commiters for the staging branch. 15:34 <stgraber> It's planned that during Plumbers 2013 next week, Daniel will hand over maintenance of the LXC project to Serge and I. 15:34 <stgraber> This should make it a lot easier for Serge and I to release new snapshots and try to move the project to a single place (likely github), reducing the amount of infrastructure we need to maintain for the project and hopefully making it easier for our contributors. 15:36 <stgraber> Some quick stats for the 1.0 release, we've had so far 343 commits from 27 different authors. We're doing around 25-30 code reviews a week which results in around 20 commits to the staging branch. 15:36 <stgraber> Canonical (through Serge, Scott and I) represent around 55% of the changes to LXC, Oracle is next with Dwight Engen contributing almost 20% of the changes, then we have a set of recuring contributors contributing around 15% of the changes and the remaining 10% are drive-by contributions. 15:36 <slangasek> stgraber: so why draw the line at 2.6.32? 15:36 <slangasek> that seems old :) 15:37 <stgraber> that's lucid and older RHEL6 15:37 <stokachu> i read that post from smoser on the overlayfs and user-data, can't wait for that in a supported release 15:37 <stgraber> (well, all RHEL6 is 2.6.32 but they cherry-picked a lot of stuff from later releases) 15:37 <slangasek> yeah - is it necessary to support lucid from lxc 1.0? I assume we aren't going to SRU it 15:37 <stgraber> also the device I use to test LXC on Android only runs 2.6.32 :) 15:38 <slangasek> hah, ok 15:38 <stgraber> LXC uses good old mailing-list posts for code reviews and we don't run any Jenkins or similar CI infrastructure. We however have a build and test environment which we use to generate our PPA builds and that runs test builds on amd64, i386 and armhf, as well as cross-compile to current Android and uploads to coverity. The result can be found at: http://lxc.dev.stgraber.org with successful builds ending up on ppa:ubuntu-lxc/daily (built for 15:38 <stgraber> Serge also maintains a few test kernels with user namespaces enabled in ppa:ubuntu-lxc/kernel 15:38 <stgraber> hmm, that first line almost certainly got cut, can someone tell me where it did? 15:39 <stokachu> (built for 15:39 <slangasek> yeah, ^^ 15:39 <stgraber> "(built for precise, quantal, raring and saucy)." 15:40 <stgraber> So as I said earlier, we're planning on releasing LXC 1.0 in early February 2014 just in time for the 14.04 FeatureFreeze. 15:40 <stgraber> From that point on, our plan is to push any fix that should get into Ubuntu to the upstream 1.0.x stable branch and get an MRE to land those directly as SRU. 15:40 <stgraber> That should be a huge improvement over our current SRU workflow where we're cherry-picking fixes one by one then SRU to 3 releases (we're pretty good at getting test results so it's not been a huge problem, but cherry-picking the fixes is time consuming). 15:41 <stgraber> LXC got promoted to main earlier in the saucy cycle and is now a supported server technology. 15:41 <stgraber> It's also heavily used by Ubuntu Touch to run the Android system on the phones and tablets where we simply boot an image of an Android system partition as the rootfs for an LXC container while sharing the network namespace with the host. 15:41 <stgraber> This gives the impression to Android that it's the only operating system running, letting it do all the hardware initialization that we need while also letting the Ubuntu system outside the container access any file or socket that we need from Android. 15:41 <stgraber> The reverse also exists with the Android/bionic port of LXC which allows running LXC natively on Android, provided the device has a kernel supporting all the features we need. This is used as the base for Ubuntu for Android. 15:43 <stgraber> Oh, and as for LXC in saucy, we're currently on a patched version of 0.9.0 but with LXC 1.0 alpha1 released yesterday, we're actively working on landing that in the archive as soon as we fix a small regression preventing LXC from running on read-only systems (a bit of a problem for Ubuntu Touch apparently). 15:43 <stgraber> So I think that's about it. I hope not too many of you felt asleep due to my rather long paste ;) 15:43 <stgraber> Any questions? 15:43 <stokachu> cant wait! 15:43 <stokachu> with the userspace stuff will be effectively replacing other virtualization technologies? 15:44 <cjwatson> stgraber: Do you have plans for further arkose work? 15:44 <cjwatson> Oh, and what's the state of LXC in Debian? 15:44 <stgraber> cjwatson: yes, I plan on reworking arkose quite a bit once we get full user namespace support in our kernel and in LXC 15:44 <stgraber> cjwatson: with the goal of having arkose run entirely as a user 15:45 <barry> stgraber: would it make sense to get the py3 bindings up on pypi? i don't think any of these are official upstream: https://pypi.python.org/pypi?%3Aaction=search&term=lxc&submit=search 15:45 <xnox> stokachu: e.g. user namespace support would it be something one would want to support natively in upstart - e.g. start this job under modified usernamespaces? (similar how we already support chroot, uid, gid stanzas) 15:45 <xnox> stgraber: e.g. user namespace support would it be something one would want to support natively in upstart - e.g. start this job under modified usernamespaces? (similar how we already support chroot, uid, gid stanzas) 15:46 <stgraber> stokachu: the user namespace work will allow users to run containers without requiring sudo rights and still get uid 0 in the container. So that should make lxc even more popular than it's today by reducing the rights an administrator has to grant. 15:46 <xnox> ... or one should simply start lxc container and go from there. 15:46 <barry> stgraber: that will be very cool 15:46 <stokachu> stgraber: awesome that was a huge feature i was hoping would make it 15:47 <stgraber> xnox: I had that discussion with jodh the other day and I think it may be interesting to allow upstart to clone a job in a new namespace, so that someone could decide that a job should get its own pid namespace or network namespace or user namespace ... as a way to reduce the attack surface 15:48 <slangasek> stgraber: and the user namespace still requires some privileged helper to give you mapped uids from a certain range, right? The kernel doesn't have a concept of "nested" uids? 15:48 <stgraber> barry: I'd have to check how pypi works but yeah, none of those are official upstream bindings and I suspect they're all wrappers around our binaries rather than binding against our library 15:48 <xnox> slangasek: that's done in passwd (define allowed mappings) and that has landed in saucy now. 15:48 <barry> stgraber: happy to help with any of that 15:49 <slangasek> xnox: yes, my point is that user namespaces work slightly differently than how one might expect 15:49 <stgraber> slangasek: right, a user needs a UID and GID range assigned in /etc/subuid and /etc/subgid. Those then get set in the user's session parent PID so that any sub process can then switch to any of those uids/gids 15:49 <slangasek> because they use up "real" uids from the root system 15:49 <xnox> slangasek: right, so it is indeed unlike pid ns. 15:50 <stgraber> then each container has a map of real uids => fake uids and real gids => fake gids and we've got tools to "switch uids/gids" 15:50 <xnox> slangasek: which does use "real" and "fake" pids at the same time.... but I see your point. 15:50 <stgraber> the containers then get their own range (subset of the parent's) so we can still do nested containers, it just requires a fair amount of configuration and enough uids/gids to begin with 15:51 * slangasek nods 15:52 <stgraber> the current blocker for having user namespace by default in our kernel is a conflict with the xfs module 15:52 <stgraber> but apparently Oracle has been working on that, so this may have been fixed already or is about to be 15:53 * stgraber checks on kernel.org 15:53 <slangasek> interesting 15:53 <stgraber> doesn't appear to be there yet, so hopefully will be in the next one 15:53 <slangasek> architecturally, the idea of using ranges of real uids still seems sketchy to me 15:54 <slangasek> I understand why it might not be realistic to have nested uids added to the kernel, since uids are everywhere :) 15:54 <slangasek> but it just feels mismatched compared with the net/fs namespaces 15:54 <stgraber> yes, it's not ideal and is pretty confusing but we had no realistic chances of getting nested uids in the kernel 15:54 * slangasek nods 15:54 <stgraber> it was already hard enough to get the rest of userns in which required changes to all the filesystem drivers and a bunch of other core changes 15:55 <stgraber> hopefully we can make our tools good enough that most users won't even have to care about it 15:55 <slangasek> so I imagine next week we're going to be talking a lot about cgroups 15:56 <slangasek> feel like explaining the mismatch between systemd and lxc cgroup heirarchies? :) 15:56 <slangasek> (if not, ok, - we're almost at time anyway) 15:56 <stgraber> sure 15:57 <stgraber> so cgroups (control groups) were designed as a way to assign resource limits and get statistics to/from processes 15:57 <stgraber> there are a variety of controllers available that let you restrict resources, get statistics or both. The current list is: 15:57 <stgraber> blkio cpu cpuacct cpuset devices freezer hugetlbmemoryperf_event 15:58 <slangasek> right - so each "controller" controls a different type of resource? 15:58 <stgraber> hmm, the last ones got messed up, those are hugetlb, memory and perf_event 15:58 <stgraber> correct 15:58 <stgraber> they work by showing a standard fs hierarchy 15:58 <stgraber> you can create directories that inherit resource allocation from their parent 15:58 <stgraber> and usually (but not necessarily) can't set allocations higher than their parent 15:59 <stgraber> most controllers also support chowning the directories to give control to non-root processes 15:59 <stgraber> using those hierarchies comes at a cost since every time the process requires access to the resource, the whole tree has to be resolved to figure out what's the actual allocation 16:00 <stgraber> so having a tree of say /systemd/user/<uid>/cgroup/... is usually seen as a bad idea (for LXC we use either /<container-name> or /lxc/<container-name> which tends to be reasonable) 16:00 <stgraber> now, logind uses cgroups but in a pretty weird way 16:01 <stgraber> it uses a fake hierarchy called "systemd" which isn't tied to any controller 16:01 <stgraber> so it's essentially cgroups without the "c" part 16:02 <stgraber> they then use the hierarchy feature to match the user and user sessions and use the notify.on_release feature of cgroups to get a notification when the cgroup is empty or they iter through /tasks to call all the pids in the cgroup 16:03 <stgraber> it's actually not so bad from a performance issue since they don't have any resource alocation applied though it's a bit of an abuse of cgroups to use a cgroup without resource controler :) 16:04 <stgraber> then there's the problem that any process running under logind will appear as part of a cgroup but only that systemd one, not any of the other. Which causes a few issues for tools expecting consistent use of cgroups (with the same hierarchy existing for all the controllers) 16:04 <slangasek> so these cgroups are actually being used exclusively for process tracking, and *not* for resource allocation? 16:04 <stgraber> for an example, look at "cat /proc/self/cgroup" on a >= raring system 16:04 <stgraber> you'll notice that the process is in an controller-less name=systemd cgroup and in / for all the others 16:05 <stgraber> now that's apparently about to change since Lennart has worked on a proposal (and I believe implementation) of a new systemd/logind API to control all the controllers 16:06 <stgraber> I don't know whether the plan is to use all of them by default on systemd systems (which would lead to a performance regression) but he's certainly pushing for libvirt and any software using cgroups to use that API for cgroup management 16:06 * slangasek nods 16:07 <slangasek> so there should be good discussions next week :) 16:07 <slangasek> stgraber: thanks for the talk 16:07 <slangasek> any questions? 16:07 * barry can't wait for the video 16:07 <stgraber> I guess so, we don't have too many (if any) talks about cgroups in the containers track as we've got more important stuff to discuss like usernamespaces but I'm sure we'll have some discussions outside the track 16:08 <stgraber> especially as Serge has been working on a similar cgroup manager before Lennart announced his plan 16:08 <slangasek> heh, alrighty :) 16:08 <stgraber> (having a privileged daemon on the host machine that containers and sub-containers can talk to to get changes applied to cgroups, doing all of the policy checks in one central place) 16:08 <slangasek> right, that was the point Lennart was making at DebConf 16:09 <slangasek> that cgroup change management needs to have a single owner 16:09 <stgraber> (that's important as containers running in a different userns won't be able to change cgroup options, so we need a privileged daemon they can talk to to do that, but I doubt logind/systemd is the answer there) 16:10 <slangasek> should be fun :) 16:10 <slangasek> #endmeeting