16:25 <blackboxsw> #startmeeting  Cloud-init bi-weekly status
16:25 <meetingology> Meeting started Mon May 13 16:25:37 2019 UTC.  The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
16:25 <meetingology> 
16:25 <meetingology> Available commands: action commands idea info link nick
16:26 <blackboxsw> hi folks, we are restarting the cloud-init community status meeting so we can have a community forum to discuss current cloud-init development and try to improve momentum in the cloud-init project efforts.
16:28 <blackboxsw> over the last couple of months we had taken a break from this meeting.  But, I think it's shown us that there is demand and usefulness in continuing hosting these meetings. So, from here on, we expect to post the upcoming date of status meetings in this channel's room topic.
16:29 <blackboxsw> Again the goal of this meeting is to give people a chance to see what work is currently being performed on upstream cloud-init as well as a forum to discuss or raise concerns or get eyes on potential bugs or features.
16:30 <blackboxsw> Generally the topics we would like to cover are the following:  (and interjections/suggestions are always welcome
16:30 <blackboxsw> topics: Previous Actions, Recent Changes, In-progress Development, Office Hours.
16:31 <blackboxsw> #topic Previous Actions
16:32 <blackboxsw> so, looking at our last status minutes from the last meeting, (on Jan 28.... :( )    we had one action carried over
16:32 <blackboxsw> #link https://cloud-init.github.io/status-2019-01-28.html#status-2019-01-28
16:33 <blackboxsw> review and land https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/356098
16:33 <blackboxsw> that's done. as well as an openstack bug...
16:33 <blackboxsw> #link https://bugs.launchpad.net/cloud-init/+bug/1801364
16:33 <ubot5`> Ubuntu bug 1801364 in cloud-init "persisting OpenStack metadata fails" [Undecided,Confirmed]
16:33 <blackboxsw> Odd_Bloke: has landed this bug, I think it needs to be marked fix committed though
16:34 <blackboxsw> Odd_Bloke: is out today at a meeting, but I'll mark that action so we can make sure that bug status is properly reported (if there are no other branches to land for it)
16:34 <blackboxsw> #action Odd_Bloke confirm correct status of https://bugs.launchpad.net/cloud-init/+bug/1801364
16:34 * meetingology Odd_Bloke confirm correct status of https://bugs.launchpad.net/cloud-init/+bug/1801364
16:34 <ubot5`> Ubuntu bug 1801364 in cloud-init "persisting OpenStack metadata fails" [Undecided,Confirmed]
16:34 <blackboxsw> That;s about it for actions from previous meeting.
16:35 <blackboxsw> #topic Recent Changes
16:36 <blackboxsw> Here we list current content that has landed in cloud-init since last meeting..... because it has been a really long time since last meeting, I'm going to redact this to the last month or so of content to avoid getting banned in this channel for spam
16:37 <blackboxsw> - Azure: Return static fallback address as if failed to find endpoint
16:37 <blackboxsw> [Jason Zions (MSFT)]
16:37 <blackboxsw> - release 19.1 (LP: #1828479)
16:37 <blackboxsw> - freebsd: add chpasswd pkg in the image [Gonéri Le Bouder]
16:37 <blackboxsw> - tests: add Eoan release [Paride Legovini]
16:37 <ubot5`> Launchpad bug 1828479 in cloud-init "Release 19.1" [Undecided,Fix released] https://launchpad.net/bugs/1828479
16:37 <blackboxsw> hah, I don't think that worked
16:37 <blackboxsw> ok breaking that content up
16:37 <blackboxsw> - Azure: Return static fallback address as if failed to find endpoint
16:37 <blackboxsw> [Jason Zions (MSFT)]
16:37 <blackboxsw> - release 19.1 (LP: #1828479)
16:37 <blackboxsw> - freebsd: add chpasswd pkg in the image [Gonéri Le Bouder]
16:37 <blackboxsw> - tests: add Eoan release [Paride Legovini]
16:37 <blackboxsw> - cc_mounts: check if mount -a on no-change fstab path
16:37 <blackboxsw> [Jason Zions (MSFT)] (LP: #1825596)
16:37 <blackboxsw> - replace remaining occurrences of LOG.warn [Daniel Watkins]
16:37 <ubot5`> Launchpad bug 1825596 in cloud-init "Azure reboot with unformatted ephemeral drive won't mount reformatted volume" [High,Fix released] https://launchpad.net/bugs/1825596
16:37 <blackboxsw> - DataSourceAzure: Adjust timeout for polling IMDS [Anh Vo]
16:37 <blackboxsw> - Azure: Changes to the Hyper-V KVP Reporter [Anh Vo]
16:37 <blackboxsw> - git tests: no longer show warning about safe yaml.
16:37 <blackboxsw> - tools/read-version: handle errors [Chad Miller]
16:37 <blackboxsw> - net/sysconfig: only indicate available on known sysconfig distros
16:37 <blackboxsw> (LP: #1819994)
16:37 <blackboxsw> - packages: update rpm specs for new bash completion path
16:37 <blackboxsw> [Daniel Watkins] (LP: #1825444)
16:37 <blackboxsw> - test_azure: mock util.SeLinuxGuard where needed
16:37 <blackboxsw> [Jason Zions (MSFT)] (LP: #1825253)
16:37 <blackboxsw> - setup.py: install bash completion script in new location [Daniel Watkins]
16:37 <blackboxsw> - mount_cb: do not pass sync and rw options to mount
16:37 <ubot5`> Launchpad bug 1819994 in Provider for Plainbox - Canonical Certification Server "cloud-init selects sysconfig netconfig renderer if network-manager is installed on Ubuntu" [Critical,Confirmed] https://launchpad.net/bugs/1819994
16:37 <blackboxsw> [Gonéri Le Bouder] (LP: #1645824)
16:37 <blackboxsw> - cc_apt_configure: fix typo in apt documentation [Dominic Schlegel]
16:37 <blackboxsw> - Revert "DataSource: move update_events from a class to an instance..."
16:37 <blackboxsw> [Daniel Watkins]
16:37 <ubot5`> Launchpad bug 1825444 in cloud-init (Ubuntu) ""make rpm" fails on CentOS after commit 947d3c208" [Undecided,Fix released] https://launchpad.net/bugs/1825444
16:37 <blackboxsw> - Change DataSourceNoCloud to ignore file system label's case.
16:37 <blackboxsw> [Risto Oikarinen]
16:38 <ubot5`> Launchpad bug 1825253 in cloud-init "Unit tests with filesystem-related mocks fail in SeLinuxGuard when run on RHEL or CentOS" [Undecided,Fix released] https://launchpad.net/bugs/1825253
16:38 <ubot5`> Launchpad bug 1645824 in cloud-init "NoCloud source doesn't work on FreeBSD" [Medium,Fix released] https://launchpad.net/bugs/1645824
16:38 <blackboxsw> - cmd:main.py: Fix missing 'modules-init' key in modes dict
16:38 <blackboxsw> [Antonio Romito] (LP: #1815109)
16:38 <blackboxsw> - ubuntu_advantage: rewrite cloud-config module
16:38 <blackboxsw> - Azure: Treat _unset network configuration as if it were absent
16:38 <blackboxsw> [Jason Zions (MSFT)] (LP: #1823084)
16:38 <blackboxsw> - DatasourceAzure: add additional logging for azure datasource [Anh Vo]
16:38 <blackboxsw> - cloud_tests: fix apt_pipelining test-cases
16:38 <ubot5`> Launchpad bug 1815109 in cloud-init "cloud-final.service: "cloud-init modules --mode final" exit with "KeyError: 'modules-init'" after upgrade to version 18.2" [Medium,Fix released] https://launchpad.net/bugs/1815109
16:38 <blackboxsw> - Azure: Ensure platform random_seed is always serializable as JSON.
16:38 <blackboxsw> [Jason Zions (MSFT)]
16:38 <blackboxsw> - net/sysconfig: write out SUSE-compatible IPv6 config [Robert Schweikert]
16:38 <blackboxsw> - tox: Update testenv for openSUSE Leap to 15.0 [Thomas Bechtold]
16:38 <ubot5`> Launchpad bug 1823084 in cloud-init "DataSourceAzure doesn't rebuild network-config after reboot" [High,Fix released] https://launchpad.net/bugs/1823084
16:38 <blackboxsw> - net: Fix ipv6 static routes when using eni renderer
16:38 <blackboxsw> [Raphael Glon] (LP: #1818669)
16:38 <ubot5`> Launchpad bug 1818669 in cloud-init "ipv6 static routes configured for eni are incorrect" [Undecided,Fix released] https://launchpad.net/bugs/1818669
16:38 <blackboxsw> - Add ubuntu_drivers config module [Daniel Watkins]
16:38 <blackboxsw> - doc: Refresh Azure walinuxagent docs [Daniel Watkins]
16:38 <blackboxsw> - tox: bump pylint version to latest (2.3.1) [Daniel Watkins]
16:38 <blackboxsw> - DataSource: move update_events from a class to an instance attribute
16:38 <blackboxsw> [Daniel Watkins] (LP: #1819913)
16:38 <blackboxsw> - net/sysconfig: Handle default route setup for dhcp configured NICs
16:38 <blackboxsw> [Robert Schweikert] (LP: #1812117)
16:38 <blackboxsw> - DataSourceEc2: update RELEASE_BLOCKER to be more accurate
16:38 <ubot5`> Launchpad bug 1819913 in cloud-init "cloud-init on xenial may generate network config on every boot" [Undecided,Fix released] https://launchpad.net/bugs/1819913
16:38 <blackboxsw> [Daniel Watkins]
16:38 <blackboxsw> - cloud-init-per: POSIX sh does not support string subst, use sed
16:38 <ubot5`> Launchpad bug 1812117 in cloud-init "route files are not written on SUSE distros" [Undecided,Fix released] https://launchpad.net/bugs/1812117
16:38 <blackboxsw> (LP: #1819222)
16:38 <blackboxsw> - Support locking user with usermod if passwd is not available.
16:38 <ubot5`> Launchpad bug 1819222 in cloud-init "cloud-init-per no longer works due to bashisms" [Undecided,Fix released] https://launchpad.net/bugs/1819222
16:38 <blackboxsw> - Example for Microsoft Azure data disk added. [Anton Olifir]
16:38 <blackboxsw> - clean: correctly determine the path for excluding seed directory
16:38 <blackboxsw> [Daniel Watkins] (LP: #1818571)
16:38 <blackboxsw> - helpers/openstack: Treat unknown link types as physical
16:38 <ubot5`> Launchpad bug 1818571 in cloud-init "cloud-init clean removes seed directory even when --seed is not specified" [High,Fix released] https://launchpad.net/bugs/1818571
16:38 <blackboxsw> [Daniel Watkins] (LP: #1639263)
16:38 <ubot5`> Launchpad bug 1639263 in cloud-init " cloud-init Unknown network_data link type: macvtap" [Undecided,Fix released] https://launchpad.net/bugs/1639263
16:38 <blackboxsw> - drop Python 2.6 support and our NIH version detection [Daniel Watkins]
16:38 <blackboxsw> - tip-pylint: Fix assignment-from-return-none errors
16:38 <blackboxsw> - net: append type:dhcp[46] only if dhcp[46] is True in v2 netconfig
16:38 <blackboxsw> [Kurt Stieger] (LP: #1818032)
16:38 <blackboxsw> - cc_apt_pipelining: stop disabling pipelining by default
16:38 <blackboxsw> [Daniel Watkins] (LP: #1794982)
16:38 <blackboxsw> - tests: fix some slow tests and some leaking state [Daniel Watkins]
16:38 <ubot5`> Launchpad bug 1818032 in cloud-init "sysconfig renders BOOTPROTO=dhcp even if dhcp=false in v2 network-config" [High,Fix released] https://launchpad.net/bugs/1818032
16:38 <blackboxsw> - util: don't determine string_types ourselves [Daniel Watkins]
16:38 <ubot5`> Launchpad bug 1794982 in cloud-init "drop /etc/apt/apt.conf.d/90cloud-init-pipelining in 16.04+" [Medium,Fix released] https://launchpad.net/bugs/1794982
16:39 <blackboxsw> Azure and SuSE and FreeBSD  foks have been quite a bit of work and we have a number of other community contributors to thank this round
16:41 <blackboxsw> AnhVoMSFT: jasonzio (MSFT) and robjo (SuSE) thanks again for your efforts
16:43 <blackboxsw> Also, as you can see, Canonical added someone to our ranks for cloud-init development. Odd_Bloke is a permanent addition to our team and landing a lot of solid content for cloud-init.
16:43 <blackboxsw> welcome Dan!
16:43 <AnhVoMSFT> thanks rharper Odd_Bloke powersj blackboxsw for all the reviewing and advice
16:44 <blackboxsw> From upstream side of the house, we have cut our first official upstream release of cloud-init. Version 19.1 sports 5 months of development, 24 contributors from 20 top-level domains. Details of that release are sent to our mailinglist and posted to the luanchpad milestone
16:44 <blackboxsw> #link https://launchpad.net/cloud-init/+milestone/19.1
16:46 <blackboxsw> :) ! it's nice how involved your teams have been thanks again.
16:46 <blackboxsw> #topic In-progress Development
16:47 <blackboxsw> So, for the upstream release of 19.1:     Canonical has just started our SRU (Stable Release Update) process where we validate the upstream release before pushing it back into earlier Ubuntu series
16:47 <blackboxsw> We follow this upstream release process for Ubuntu
16:47 <blackboxsw> #link https://wiki.ubuntu.com/CloudinitUpdates
16:48 <blackboxsw> I've queue to release branches for review as of Friday 05/10. I have queued Ubuntu Disco and published to disco-proposed pocket already.
16:48 <blackboxsw> So if someone deploys disco on your favorite cloud or hardware, you can add a disco-proposed apt source to /etc/apt/sources.list and you sound be able to install and or test 19.1
16:49 <blackboxsw> I made the 19.1.1 SRU cut which included the commitish baa478546d8cac98a706010699d64f8c2f70b5bf  which was the tip containing " Azure: Return static fallback address as if failed to find endpoint"  as the last commit
16:49 <blackboxsw> so jasonzio's latest azure branch made it
16:50 <blackboxsw> expectation is that I'll get most of verification done over this week and we can hopefully (pending no SRU regressions discovered) publish to Xenial, Bionic, Cosmic and Disco the beginning of next week (05/20/2019)
16:51 <blackboxsw> I just added the following card to track this work, it'll move to DONE when published
16:51 <blackboxsw> #link https://trello.com/c/oxv61bPl/1099-sru-191-xenial-bionic-cosmic-and-disco
16:51 <powersj> AnhVoMSFT, thank you for all your contributions
16:52 <AnhVoMSFT> is there a package available publicly for 19.1 that we can download and apply it to existing images and run through our perf. regression test?
16:52 <blackboxsw> also today, we'll post 19.1 to our copr el-testing repo at
16:52 <blackboxsw> #link https://copr.fedorainfracloud.org/coprs/g/cloud-init/el-testing/
16:52 <blackboxsw> for centos.
16:53 <powersj> AnhVoMSFT, daily PPA: https://launchpad.net/~cloud-init-dev/+archive/ubuntu/daily
16:53 <blackboxsw> AnhVoMSFT: we have daily repos that
16:53 <blackboxsw> thx powersj
16:53 * powersj disappears
16:54 <blackboxsw> AnhVoMSFT, so that is built from tip of master (which currently contains exactly the same commit set as will be published to each series)
16:54 <blackboxsw> AnhVoMSFT: once the following branches are "merged" for each series, xenial-proposed, bionic-proposed, cosmic-proposed will also contain the final bits for each series
16:54 <AnhVoMSFT> thanks powersj blackboxsw
16:55 <blackboxsw> #link https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/367301
16:55 <blackboxsw> #link https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/367302
16:55 <blackboxsw> #link https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/367297
16:55 <blackboxsw> so when we land those, it should be a matter of an hour or so until <series>-proposed is accessible to test those final packages
16:56 <blackboxsw> AnhVoMSFT: in the SRU process, we still will have to verify and test those bits before release. There is a tracking bug for the overall process here
16:56 <blackboxsw> #link https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1828637
16:56 <ubot5`> Ubuntu bug 1828637 in cloud-init (Ubuntu) "sru cloud-init (18.5-45 to 19.1.1) Xenial, Bionic, Cosmic, Disco" [Undecided,In progress]
16:56 <blackboxsw> ^ this but will go to fix released when we pass verification and publish the SRU
16:57 <blackboxsw> which will be promotion from xenial-proposed to xenial-updates for public availability on all images
16:57 <blackboxsw> again the estimate on this is probably next week for getting through testing
16:58 <blackboxsw> the rest of the cloud-init team is meeting this week for feature planning for the next cycle. But generally  we will be tracking our ongoing work @ our public trello board
16:59 <blackboxsw> #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin
17:00 <blackboxsw> If there are there any features or significant gaps that people have interest in that we could try to put on our radar feel free to drop us a line here in channel or on the mailinglist @ cloud-init@lists.launchpad.net
17:01 <blackboxsw> I think that about wraps up current in-flight work
17:02 <blackboxsw> #topic Office hours (next ~30 mins)
17:03 <blackboxsw> This section is allocated generally for folks to chat about any burning questions that we have. Bring your topics, bugs, feature requests, merge proposals or questions and we'll try getting some eyes or discussion on the
17:03 <blackboxsw> *them.
17:04 <blackboxsw> Generally we should have a couple cloud-init devs hanging out in channel during this time to help build momentum or unblock folks. Today may be a bit light in attendance due to a remote work trip a number of people are attending.
17:04 <AnhVoMSFT> if our next work requires importing another module (python-dbus), are there any guidelines wrt to using additional python modules not currently part of cloudinit package?
17:06 <blackboxsw> AnhVoMSFT: generally adding new dependencies is something we try to avoid if possible due to impacting the minimal cloud images with additional dependencies. From ubuntu's perspective we tend to try to look at an ubuntu minimal image to see if the dependency is already there (and that makes things simple for the addition).
17:06 <blackboxsw> If new dependency is also something that can be optional instead of strict it makes adding that dependency easier.
17:06 <blackboxsw> For example: python-jsonschema is an optional package dependency
17:07 <blackboxsw> we bookend the imports in cloud-init with a try/except ImportError: pass and make sure that cloud-init still behaves if it is not there.
17:07 <AnhVoMSFT> i will have to look into how to make it optional - certainly we can go that way if that makes thing easier
17:07 <AnhVoMSFT> what we are looking at is how to get the kernel init timestamp and systemd init timestamp - both of these are available as KernelTimestamp and UserSpaceTimestamp that are available as part of the systemd's system dbus
17:07 <blackboxsw> This type of approach has been helpful in supporting Centos 6 where jsonschema wasn't available in stock images
17:08 <AnhVoMSFT> normally you can get those info from "systemctl show", but the issue is systemctl internally use systemd's format_timestamp, which trims the precision down to "seconds" level. (format_timestamp_us would give us what we need)
17:11 <blackboxsw> AnhVoMSFT: interesting. since it seems like the dependency is one one or two values and not the full content of python-dbus library/modules I wonder if it might be better to see if those values are available in /proc or /sys somewhere as it might be fairly simple to instrument our own. (and avoid the time cost of importing larger python modules)
17:11 <AnhVoMSFT> I guess if we really want to avoid another module, we can use os.stat to key off ctime of /proc and /proc/1 - however, the ctimes of /proc and /proc/1 are a bit earlier than the actual KernelTimestamp and UserSpaceTimestamp. They do have the advantage of being more widely available. Systemd and dbus might not exist on all distros
17:11 <blackboxsw> ... at least something to peek at to see if it is low hanging fruit that is easy to handle
17:12 <AnhVoMSFT> Right, I would appreciate if someone with more knowledge on Linux systems can provide some advices here
17:12 <AnhVoMSFT> the dependency is really on those values, and the dbus library seems a bit overkill for it
17:13 <blackboxsw> rharper: Odd_Bloke may have some thoughts on that too. but that conversation may have to wait a bit due to time difference.
17:14 <blackboxsw> AnhVoMSFT: again, what call/values would you like to make/get from python-dbus just for clarity
17:15 <AnhVoMSFT> bus=dbus.SystemBus() systemd1=bus.get_object('org.freedesktop.systemd1','/org/freedesktop/systemd1') manager=dbus.Interface(systemd1, 'org.freedesktop.systemd1.Manager') props_iface=dbus.Interface(manager,'org.freedesktop.DBus.Properties') props_iface.Get('org.freedesktop.systemd1.Manager','KernelTimestamp')
17:15 <AnhVoMSFT> oops - multiline didn't seem to go so well
17:15 <blackboxsw> it's all good thx
17:16 <blackboxsw> alternately, I support we could shell out via util.subp to make a cli call to handle that to avoid a python package dependency, but again it'd probably be better/faster to avoid that
17:17 <AnhVoMSFT> right, we're trying to avoid adding more util.subp call (in fact we have a workitem to reduce the numbers of the existing util.subp call the Azure DS makes)
17:18 <AnhVoMSFT> Like mentioned previously, we're looking at the /proc and /proc/1 ctime, they're cheap using the os.stat module and exist in all distros.
17:19 <blackboxsw> oops typo "alternatively , I support" ///   I suppose
17:19 <blackboxsw> yeah that seems like the most non-contentious approach (processing /proc/1 ctime)
17:20 <blackboxsw> uncontentious rather
17:22 <AnhVoMSFT> another thing we're looking at is cloud-init cloudtests so that we can have some azure cloud test running as part of cloud-init check-in gate
17:22 <blackboxsw> excellent!
17:23 <AnhVoMSFT> if you guys have any pointers/tips on what it takes to enable cloud-tests please let us know
17:23 <AnhVoMSFT> one of our team members will be looking at it
17:24 <blackboxsw> any blockers that you are aware of to date?  Generally the biggest lift is getting the platform support added to the cloud-tests
17:25 <blackboxsw> I know powersj added more CI docs related to this a few months ago at
17:25 <blackboxsw> #link https://cloudinit.readthedocs.io/en/latest/topics/tests.html#
17:25 <blackboxsw> I'll try to see if there was additional documentation tossed around
17:26 <blackboxsw> the benefit if Azure platform is added to cloud-init tip is that'd we would likely add some of that testing to our SRU verification process each release we are doing. I'm excited about that
17:27 <blackboxsw> right now, it's manual for Azure testing... per something like this https://github.com/cloud-init/ubuntu-sru/blob/master/manual/azure-sru-18.5.21.txt
17:27 <AnhVoMSFT> I'm not aware of any blockers, we just barely started looking at it
17:27 <AnhVoMSFT> these links are very helpful, I'll pass them along
17:28 <blackboxsw> the above script is tedious to write up, and only is really testing upgrade path and any *new* azure-specific bugs/features that are fixed in the current SRU, since most of the other integration testing covers general case
17:28 <blackboxsw> thanks AnhVoMSFT that's good to hear
17:29 <blackboxsw> are there any other topics folks have for today's meeting? Otherwise we can adjourn until next meeting
17:29 <blackboxsw> next meeting will be May 27th, same time
17:30 <blackboxsw> and I'll post status notes for the meeting to our github pages
17:30 <blackboxsw> #link https://cloud-init.github.io
17:31 <blackboxsw> before I forget. let's get an action to followup on kernel/userspace timestamp discussion
17:31 <AnhVoMSFT> thanks blackboxsw
17:32 <blackboxsw> #action Odd_Bloke/rharper/AnhVoMSFT followup on recommended approach for getting kernel/userspace timestamp with or without python-dbus package dependency
17:32 * meetingology Odd_Bloke/rharper/AnhVoMSFT followup on recommended approach for getting kernel/userspace timestamp with or without python-dbus package dependency
17:32 <blackboxsw> thank you AnhVoMSFT
17:32 <blackboxsw> see folks next time, or anytime in channel
17:32 <blackboxsw> #endmeeting