16:05 <blackboxsw> #startmeeting Cloud-init bi-weekly status meeting 16:05 <meetingology> Meeting started Tue May 29 16:05:51 2018 UTC. The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology. 16:05 <meetingology> 16:05 <meetingology> Available commands: action commands idea info link nick 16:06 <dpb1> dmbaturin: there will be a time for open questions in this meeting in just a few minutes. :) 16:06 <dpb1> so you will have the right people around 16:06 <blackboxsw> welcome folks to another cloud-init community status meeting, today's meeting delayed by one day due to US holiday. Next meeting will be June 11th. same time 16:07 <blackboxsw> I've added an actions topic to this meeting so we can wrap up or carry over any actions discussed last time 16:08 <blackboxsw> the topics will be Previous Actions, Recent Changes, In-progress Development, and Office Hours 16:08 <dmbaturin> Oh, cool. 16:09 <blackboxsw> As always notes will be posted to the following site 16:09 <blackboxsw> #link https://cloud-init.github.io/ 16:09 <blackboxsw> welcome dmbaturin good timing. :) 16:09 <blackboxsw> #topic Previous Actions 16:09 <dmbaturin> Yeah, I'm just in time it seems. ;) 16:09 <blackboxsw> 2 weeks ago we had a couple of followup items that needed some extra review: 16:10 <blackboxsw> * ACTION: blackboxsw review distro dection and empty modules list 16:10 <blackboxsw> * ACTION: robjo review existing chrony support in master per rharper's work 16:10 <blackboxsw> * ACTION: blackboxsw carryover network hotplug vs network maintenance on reboot-only 16:10 <blackboxsw> we did get through robjo's branches on distro *detection* and landed them\ 16:11 <blackboxsw> and I know our team also discussed a potential approach to network hotplug vs network maintenance to better enable SmartOs folks who want to handle network config across reboots only 16:12 <blackboxsw> I think we decided we needed to draw up a quick shared document on a proposal which would allow for maintenance on reboots only vs true hotplug. 16:12 <blackboxsw> I'll carry over that action to write up a doc on this and send it to list by the next meeting 16:13 * blackboxsw blackboxsw write up short doc/branch on hotplug versus network maintenance on reboot for comment 16:13 <blackboxsw> and I believe robjo from SuSE was able to get through rharper's chrony support branch with a couple comments too 16:13 <blackboxsw> so no other actions from last meeting 16:13 <blackboxsw> #topic Recent Changes 16:14 <blackboxsw> this following content landed in cloud init tip over the last two weeks 16:14 <blackboxsw> - Do not use the systemd_prefix macro, not available in this environment 16:14 <blackboxsw> [Robert Schweikert] 16:14 <blackboxsw> - doc: Add config info to ec2, openstack and cloudstack datasource docs 16:14 <blackboxsw> [Chad Smith] 16:14 <blackboxsw> - Enable SmartOS network metadata to work with netplan via per-subnet 16:14 <blackboxsw> routes [Dan McDonald] (LP: #1763512) 16:14 <blackboxsw> - openstack: Allow discovery in init-local using dhclient in a sandbox. 16:14 <ubot5> Launchpad bug 1763512 in cloud-init "DataSourceSmartOS ignores sdc:routes" [Medium,Fix committed] https://launchpad.net/bugs/1763512 16:14 <powersj> lol! 16:14 <powersj> welcome back 16:14 <blackboxsw> heh looks like I got kicked for the paste :) 16:14 <powersj> blackboxsw: your last message was - openstack: Allow discovery in init-local using dhclient in a sandbox. 16:15 <blackboxsw> - tests: Avoid using https in httpretty, improve HttPretty test case. 16:15 <blackboxsw> 10:14 (LP: #1771659) 16:15 <blackboxsw> 10:14 - yaml_load/schema: Add invalid line and column nums to error message 16:15 <blackboxsw> 10:14 [Chad Smith] 16:15 <blackboxsw> 10:14 - Azure: Ignore NTFS mount errors when checking ephemeral drive 16:15 <blackboxsw> 10:14 [Paul Meyer] 16:15 <ubot5> Launchpad bug 1771659 in cloud-init "unittests fail in OpenSuSE 42.3 with httpretty issues" [Medium,Fix committed] https://launchpad.net/bugs/1771659 16:15 <blackboxsw> - packages/brpm: Get proper dependencies for cmdline distro. 16:15 <blackboxsw> 10:14 - packages: Make rpm spec files patch in package version like in debs. 16:15 <blackboxsw> 10:14 - tools/run-container: replace tools/run-centos with more generic. 16:15 <blackboxsw> 10:14 - Update version.version_string to contain packaged version. (LP: #1770712) 16:15 <blackboxsw> 10:14 - cc_mounts: Do not add devices to fstab that are already present. 16:15 <blackboxsw> 10:14 [Lars Kellogg-Stedman] 16:15 <blackboxsw> 10:14 - ds-identify: ensure that we have certain tokens in PATH. (LP: #1771382) 16:15 <blackboxsw> 10:14 - tests: enable Ubuntu Cosmic in integration tests [Joshua Powers] 16:15 <blackboxsw> 10:14 - read_file_or_url: move to url_helper, fix bug in its FileResponse. 16:15 <blackboxsw> 10:14 - cloud_tests: help pylint [Ryan Harper] 16:15 <blackboxsw> 10:14 - flake8: fix flake8 errors in previous commit. 16:15 <ubot5> Launchpad bug 1770712 in cloud-init "It would be nice if cloud-init provides full version in logs" [Medium,Fix committed] https://launchpad.net/bugs/1770712 16:15 <blackboxsw> 10:14 - typos: Fix spelling mistakes in cc_mounts.py log messages [Stephen Ford] 16:15 <blackboxsw> 10:14 - tests: restructure SSH and initial connections [Joshua Powers] 16:15 <ubot5> Launchpad bug 1771382 in cloud-init "ds-identify: fails to recognize NoCloud datasource on boot cause it does not have /sbin in $PATH and thus does not find blkid" [Low,Fix committed] https://launchpad.net/bugs/1771382 16:15 <blackboxsw> 10:14 - ds-identify: recognize container-other as a container, test SmartOS. 16:15 <blackboxsw> ok hopefully we ended on ds-identify 16:16 <dmbaturin> Yes, we did. 16:16 <blackboxsw> excellent. sorry for the paste, I'll send this out to cloud-init@lists.canonical.com a day before the next meeting so we don't have to IRC flood here 16:17 <blackboxsw> make that cloud-init@lists.launchpad.net 16:17 <blackboxsw> also we finished our SRU (stable release update) of cloud-init 18.2.27 to Bionic. 16:18 <blackboxsw> Ubuntu Cosmic currently reflects near tip of master 18.2.59 16:18 <blackboxsw> ok that's all for Recent Changes 16:18 <blackboxsw> anything I'm missing powersj ? 16:19 <powersj> I think you are good 16:19 <blackboxsw> #topic In-progress Development 16:19 <blackboxsw> We track upstreams progress publicly in trello 16:19 <blackboxsw> #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin 16:20 <blackboxsw> any blue labeled cards are cloud-init core work 16:21 <blackboxsw> we have been fixing a couple of bugs raised by our CI infrastructure on newer series of Ubuntu . currently a minor issue with salt minion on Bionic or later, and a couple of unit and integration test race conditions 16:23 <blackboxsw> big ticket items for cloud-init in the nearterm are metadata standardization across clouds, so cloud-init scripts/cloud-config template can source these cloud-provided values 16:24 <dmbaturin> Metadata standardization is something I really would like to see, if you need more hands for that, let me know. 16:24 <blackboxsw> the standardization of this instance-data will allow folks to script against any standard values provided to cloud-init in the same way on any cloud. Think hostname, fqdn, ip addrs, region name etc. 16:24 <dmbaturin> SSH keys too! 16:24 <blackboxsw> definitely dmbaturin I'll point you at a couple branches and what we're thinking 16:25 <blackboxsw> this conent will show up in /run/cloud-init/instance-data.json 16:25 <blackboxsw> #link https://cloudinit.readthedocs.io/en/latest/topics/datasources.html?highlight=instance-data#instance-data 16:26 <blackboxsw> and will also be referenced via jinja template variables 16:26 <blackboxsw> and a cloud-init query CLI 16:28 <dmbaturin> The data will be updated whenever a change in the environment is made? 16:28 <blackboxsw> Also powersj will be working toward a common library for cloud-testing in the weeks to come which cloud-init integration tests will leverage to drive lxd, ec2, openstack azure etc for a cloud testing 16:29 <dmbaturin> Also, will it be possible to stop cloud-init from doing anything but writing that data and starting an external script to process it? 16:30 <blackboxsw> dmbaturin: some of that functionality will be handled in the hotplug work we are starting on. There will be operations that can be triggered by either a hotplug monitor on metadata or by cloud-init's CLI to say query from cache (the instance-data.json file) versus query fresh/update 16:30 <dmbaturin> I see. 16:33 <blackboxsw> dmbaturin: cloud-inits init-local or init-network stage is what calls "get_data" on the give datasources to collect and write that data to file. Spawning a script is generally done through runcmd which happens in cloud-init's 'final' stage. Trying to decouple them (and skipping the modules:config stage) is possible by altering /etc/cloud/cloud.cfg in a custom image to specify no modules in a given stage. Though it's not 16:33 <blackboxsw> really recommeded as most of the modules only do a quick sanity check to see if they are specifically enabled before trying to do any realy work 16:33 <blackboxsw> we try to keep boot time as fast as possible and cut out the fat where we can 16:33 <blackboxsw> if that's the concern you had 16:34 <blackboxsw> ok I think that's it for In-progress development we can move to office hours for all addtional discussion 16:34 <blackboxsw> #topic Office Hours (next ~30 mins) 16:35 <dmbaturin> No, boot time is not the primary concern here, my concern is how to ensure no module is trying to treat our system as if it was a normal Debian (which either doesn't work or can potentially get the system into an inconsistent state). 16:36 <dmbaturin> I guess if we are having a real meeting, it may be a good idea to formally introduce myself and the project. :) 16:36 <blackboxsw> All topics of interest to cloud-init development can be brought up and discussed here. If there are merge proposal that need attention, bugs that need work just bring them up here we should have a few sets of eyes on this channel to discuss and comment 16:36 <blackboxsw> sounds good dmbaturin introduce away :) 16:37 <blackboxsw> Chad smith, Canonical, one of the maintainers of cloud-init. We have a few others here (some on vacation). powersj rharper smoser dpb1 all canonical as well. 16:38 <dpb1> idk who blackboxsw is 16:38 <dpb1> he might be crazy 16:38 <blackboxsw> frequently we have other distribution developers and cloud devs here too (SuSE, RedHat, Microsoft Azure, SmartOS, VMWare ) 16:38 <blackboxsw> heh, I'm just a bot 16:38 <dmbaturin> So, I'm one of the maintainers of the VyOS project (http://vyos.io). It's a distro for routers and firewalls whose primary goal is to be just like hardware routers, but not tied to any hardware, which includes a single config file and unified CLI with a commit/rollback model, versioning, and cross-checks (e.g. if you try to reference a non-existent NIC in DHCP configuration, commit fails). 16:39 <dpb1> nice to meet you dmbaturin 16:40 <blackboxsw> ahh makes sense. So debian-based os kindof, which is why you'd want to lock down what modules run. 16:40 <dmbaturin> We support all major virtualization platforms now in the sense of including all required drivers and utilities, but autoconfiguration on cloud platforms is only supported for EC2 via a custom script, so we are looking to ways to support more clouds, ideally without doing the work that is already done, or at least contributing those general things into something where more people can benegit from it, not just us. 16:41 <blackboxsw> alos dmbaturin each config module claims what distro is supported in a distro property, so you could vet what modules you want to run, and only add VyOS to the list of compatible distros. Config modules all live in source at cloudinit/config/cc_*py. 16:41 <blackboxsw> but we can discuss that confuig module support (or not) once you dig in to look at supporting VyOS 16:42 <dmbaturin> Yes, I'm thinking how exactly it should be done. 16:42 <blackboxsw> dmbaturin: cloud-init's a pretty good choice for getting that cloud-support breadth for free 16:43 * robjo sorry I'm late 16:43 <blackboxsw> robjo: sorry for the late change from yesterday's normal meet time 16:43 <dmbaturin> The least intrusive option would be to indeed improve the instance data format, so that we can simply pass it to our own script, which is why I'm all for contributing to it. 16:43 <robjo> blackboxsw: noLnxDistro branch has not yet been merged 16:44 <blackboxsw> bah robjo ahh you're right 16:44 <blackboxsw> ok looks like you handled all review comments. I'll get it landed today 16:44 <smoser> powersj: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/347060 16:44 * blackboxsw blackboxsw land https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/336794 today 16:45 <smoser> you moved that to 'approved' i guess ? 16:45 <smoser> which meant the bot didnt comment (sorry blackboxsw ... interupted) 16:45 <dmbaturin> blackboxsw: Could you point me to the branches were the work on instance data is going on? 16:45 <blackboxsw> dmbaturin: here's one stale one I need to get back to this week. for enabling the template reference of instance-data.json content 16:46 <powersj> smoser: ah sorry you are right 16:47 <smoser> powersj: i'm going to land it anyway 16:47 <robjo> also I think emptyStageOK branch should be ready to go 16:47 <powersj> ok 16:47 <blackboxsw> hrm digging on the metadata branch. 16:48 <blackboxsw> dmbaturin: the trello card I'll be tying branches to is this one 16:48 <blackboxsw> #link https://trello.com/c/5n5B8x23/802-cloud-init-query-standardized-json-information 16:48 <robjo> and https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/333904 should be on smoser plate 16:49 <robjo> or anyone else who wants to pick it up and get it merged, please 16:49 <blackboxsw> #link https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/335290 16:50 <blackboxsw> ^ dmbaturin initial template handling thoughts.... I have to create at least 2 branches to standardize datasource class apis to make the metadata content easier to generalize and I can add your launchpad username to the review as I put them up 16:50 <rharper> smoser: re: hostname, yes, that's right; we probably could update the set_hostname docs to mention that detail w.r.t early hostname setting 16:51 <rharper> blackboxsw: sorry to interrupt 16:51 <blackboxsw> dmbaturin: what's your launchpad user name? (mine's chad.smith) 16:51 <dmbaturin> blackboxsw: dmbaturin 16:51 <blackboxsw> heh. 16:51 <blackboxsw> thx 16:51 <dmbaturin> I'm too predictable. ;) 16:51 <blackboxsw> yeah, I lost a bet on blackboxsw :) 16:51 <blackboxsw> robjo: okay adding that branch too for review/landing 16:51 * blackboxsw land https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/333904 16:52 * blackboxsw land https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/336794 16:53 <blackboxsw> alright, any other topics or discussions? 16:53 <dmbaturin> Trello is integrated with launchpad? 16:55 <blackboxsw> dmbaturin: nope, just easy to use for our agile workflow. And simple to cut-paste links, assign people, drag to different lanes as the work progresses 16:55 <blackboxsw> has a lot of github integraiton if you get the right plugins 16:56 <blackboxsw> we have some minimal tooling that can talk to lauchpad and inject cards, but that's hand-written, not part of trello product. 16:59 <dmbaturin> I mean, if you add my username there, will I get any notifications about card changes. 16:59 <blackboxsw> dmbaturin: I get emails from all trello card moves,changes. let's see 16:59 <blackboxsw> I can subscribe you to the card (you want the standardized json stuff?) 17:00 <dmbaturin> Yes. 17:02 <blackboxsw> hrm can't find your user 17:02 <blackboxsw> ahh 17:03 <blackboxsw> I think I invited you 17:03 <blackboxsw> and added your user to the card so you can watch it progress 17:05 <blackboxsw> ok I think that about wraps up our meeting for today 17:05 <blackboxsw> any parting shots? 17:05 <blackboxsw> I'll post these notes to our github project page 17:05 <blackboxsw> #link https://cloud-init.github.io/ 17:05 <blackboxsw> thanks again all 17:05 <blackboxsw> #endmeeting