16:08 <blackboxsw> #startmeeting Cloud-init bi-weekly status meeting
16:08 <meetingology> Meeting started Mon Jan 22 16:08:22 2018 UTC.  The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
16:08 <meetingology> 
16:08 <meetingology> Available commands: action commands idea info link nick
16:08 <blackboxsw> certianly ajorg :) (on office hours)
16:09 <blackboxsw> Welcome to another episode of cloud-init bi-weekly status. We'll chat about about cloud-init updates and in progress work, and we'l drop into office hours for ongoing discussions/bug work etc.
16:10 <blackboxsw> #topic Recent changes
16:11 <blackboxsw> Just walking through git-log for what we have committed in the last couple of weeks, here's the brief summary
16:12 <blackboxsw> thx smoser
16:12 <blackboxsw> - shorten the message in the exception per powersj feedback
16:12 <blackboxsw> - Use the same botocore session so the patched changes stick.
16:12 <blackboxsw> - fix bad use of %
16:12 <blackboxsw> - Fix console_log, improve comments and raise PlatformError on.
16:12 <blackboxsw> - tests: Fix EC2 Platform to return console output as bytes.
16:12 <blackboxsw> - tests: remove zesty as supported OS to test [Joshua Powers]
16:12 <blackboxsw> - Do not log warning on config files that represent None. (LP: #1742479)
16:12 <ubot5> Launchpad bug 1742479 in cloud-init (Ubuntu) "setting manual_cache_clean causes warning" [Medium,Fix released] https://launchpad.net/bugs/1742479
16:12 <blackboxsw> - tests: Use git hash pip dependency format for pylxd.
16:12 <blackboxsw> - tests: add integration requirements text file [Joshua Powers]
16:12 <blackboxsw> - MAAS: add check_instance_id based off oauth tokens. (LP: #1712680)
16:12 <blackboxsw> - tests: update apt sources list test [Joshua Powers]
16:12 <blackboxsw> - tests: clean up image properties [Joshua Powers]
16:12 <blackboxsw> - tests: rename test ssh keys to avoid appearance of leaking private keys.
16:12 <ubot5> Launchpad bug 1712680 in maas-images "cloud-init re-generates network config every reboot overwriting manual admin changes on CentOS." [Undecided,New] https://launchpad.net/bugs/1712680
16:12 <blackboxsw> [Joshua Powers]
16:12 <blackboxsw> - tests: Enable AWS EC2 Integration Testing [Joshua Powers]
16:12 <blackboxsw> - cli: cloud-init clean handles symlinks (LP: #1741093)
16:12 <ubot5> Launchpad bug 1741093 in cloud-init "cloud-init clean traceback on instance dir symlink" [Low,Fix committed] https://launchpad.net/bugs/1741093
16:13 <ajorg> What's being patched in botocore?
16:13 <blackboxsw> So a number of changes went into integration test related work, separating out requirements files.
16:14 <blackboxsw> MAASDatasource now also has smarted cache handling based on oauth token renewal from the maas server
16:14 <blackboxsw> so botocore is used by integration tests only as a mechanism to talk to the instance under test... looking back at the specifics here
16:14 <blackboxsw> it might have just been shuffling out how and where we define the dependency
16:14 <smoser> blackboxsw: (my 'paste' to you was bad... http://paste.ubuntu.com/26438113/ is better, showing only those on master, not my local branch that was currently checked out )
16:15 <blackboxsw> heh, oopsie daisy let's paste again inline then
16:15 <blackboxsw> - tests: remove zesty as supported OS to test [Joshua Powers]
16:15 <blackboxsw> - Do not log warning on config files that represent None. (LP: #1742479)
16:15 <blackboxsw> - tests: Use git hash pip dependency format for pylxd.
16:15 <blackboxsw> - tests: add integration requirements text file [Joshua Powers]
16:15 <blackboxsw> - MAAS: add check_instance_id based off oauth tokens. (LP: #1712680)
16:15 <blackboxsw> - tests: update apt sources list test [Joshua Powers]
16:15 <blackboxsw> - tests: clean up image properties [Joshua Powers]
16:15 <blackboxsw> - tests: rename test ssh keys to avoid appearance of leaking private keys.
16:15 <blackboxsw> [Joshua Powers]
16:15 <blackboxsw> - tests: Enable AWS EC2 Integration Testing [Joshua Powers]
16:15 <blackboxsw> - cli: cloud-init clean handles symlinks (LP: #1741093)
16:15 <ubot5> Launchpad bug 1742479 in cloud-init (Ubuntu) "setting manual_cache_clean causes warning" [Medium,Fix released] https://launchpad.net/bugs/1742479
16:15 <ubot5> Launchpad bug 1712680 in maas-images "cloud-init re-generates network config every reboot overwriting manual admin changes on CentOS." [Undecided,New] https://launchpad.net/bugs/1712680
16:15 <ubot5> Launchpad bug 1741093 in cloud-init "cloud-init clean traceback on instance dir symlink" [Low,Fix committed] https://launchpad.net/bugs/1741093
16:15 <blackboxsw> ok the real deal, that looks better
16:16 <blackboxsw> ahh ajorg that interim commit message on botocore was about integration tests caching the session information during testing so we don't recreate that session with every ssh connection to the instance
16:16 <blackboxsw> just a little time savings per review comments on powersj branch I believe
16:17 <ajorg> okay, so nothing that needs to get upstreamed to botocore?
16:17 <blackboxsw> I don't think so, powersj smoser I have vague recollection of someone filing an upstream botocore issue. did we have to do that for something else though?
16:18 <powersj> https://github.com/boto/botocore/issues/1351
16:18 <powersj> that was the issue smoser put in ^
16:18 <blackboxsw> nice recall powersj thanks.
16:18 <blackboxsw> #link https://github.com/boto/botocore/issues/1351
16:20 <smoser> ajorg: you can read that bug.  imo they have a data loss error, but not one that they can easily fix without causing failures in places that previously ran fine.
16:20 <ajorg> I'll ask them to re-open it.
16:21 <ajorg> At the very least they should answer your last.
16:21 <smoser> thanks.
16:22 <blackboxsw> Generally anything significant that we have landed (and any inprogress work) should be available at the following link.
16:22 <blackboxsw> #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin
16:23 <blackboxsw> anything else we should note over the last couple weeks?
16:23 <blackboxsw> otherwise I'll switch to ongoing work topic
16:24 <blackboxsw> #topic In-progress Development
16:25 <blackboxsw> As you may have seen last week, we've gotten through a few passes and discussions around dojordan's branch to define pre-provisioning
16:25 <blackboxsw> #link https://code.launchpad.net/~dojordan/cloud-init/+git/cloud-init/+merge/334341
16:26 <blackboxsw> some of that discussion resulted in a new context manager: EphemeralDHCPv4 to support a sandboxed dhclient request on an instance.
16:27 <blackboxsw> this context manager affects Ec2 datasource a bit as it encapsulates all of the dhcp request -> EphemeralIPV4Network calls that Ec2 was doing
16:28 <blackboxsw> there may be a couple other datasources that follow suit with this type of sandboxed dhcp request in weeks to come
16:28 <ajorg> glad it turned out to be generally useful rather than only specifically to ec2
16:28 <blackboxsw> absolutely
16:30 <blackboxsw> Some other in-progress bits look like we might try focusing a bit more on chrony support and gettting robjo's branches some more eyes.
16:31 <blackboxsw> and some work on Ubuntu snappy support per the snappy and snap config modules.
16:31 <smoser> dojordan: i just put one comment on your mp. /me thanks dojordan again for his patience.
16:32 <blackboxsw> rharper: smoser powersj anything more in the immediate pipeline that I'm missing/
16:32 <blackboxsw> ?
16:32 <smoser> blackboxsw: we should get the EphemeralDHCP thingy into the digital ocean datasource also.
16:32 <rharper> blackboxsw: a reply to the network discussion on the list from the azure folks and robjo
16:32 <ajorg> I took another look at https://code.launchpad.net/~yeazelm/cloud-init/+git/cloud-init/+merge/331897 and saw that origin/master seems to be failing some of the integration tests too.
16:32 <ajorg> (at least for me, locally, on a 16.04 instance)
16:32 <blackboxsw> ahhh right forgot about all your work there rharper, thanks!
16:33 <smoser> ajorg: https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-ci-nightly/
16:33 <smoser> that is nigytly run of trunk
16:33 <blackboxsw> #link https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-ci-nightly/
16:34 <ajorg> I'll try blackholing IMDS on my instance. Could be that's interfering with something.
16:35 <smoser> it is red, but 218 (green) and 219 (red) used the same git has on trunk (5cc0b19b8).
16:35 <ajorg> I'll follow up during office hours
16:36 <smoser> can you give me example of your failures ? we had "disk full" errors recently on our jenkins, so that might be the cause of the issue for 291.
16:36 <smoser> s/291/219/
16:36 <blackboxsw> I don't remember seeing that traceback recently. w/ warning messages present in cloud-init
16:36 <smoser> powersj: ? can you explain lxc timeout failure at
16:36 <smoser> https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-ci-nightly/219/consoleFull
16:37 <powersj> smoser: we discovered that our qemu-migration test was installing lxd from the archive and causing conflicts with the snap installed lxd
16:37 <powersj> I have a message to christian to prevent it, and I have already cleaned it up
16:37 <powersj> so new runs should pass
16:37 <ajorg> 2018-01-22 16:19:03,550 - tests.cloud_tests - WARNING - test case: modules/ssh_import_id failed TestSshImportId.test_no_stages_errors with: AssertionError: 1 != 0 : errors ['(\'ssh-import-id\', ProcessExecutionError("Unexpected error while running command.\\nCommand: [\'sudo\', \'-Hu\', \'ubuntu\', \'ssh-import-id\', \'gh:powersj\', \'lp:smoser\']\\nExit code: 1\\nReason: -\\nStdout: -\\nStderr: -",))'] were encountered in stage m
16:38 <smoser> hm.. well, that will hit launchpad.net over https
16:38 <smoser> cloud-init-output.log probaly has more info (should be collected)
16:38 <powersj> the actual error is:   File "/var/lib/jenkins/slaves/torkoal/workspace/cloud-init-ci-nightly/tests/cloud_tests/platforms/instances.py", line 142, in _wait_for_system
16:38 <powersj> raise OSError('timeout: after {}s system not started'.format(time))
16:38 <powersj> it is because when the qemu tests installed lxd it didn't initialize lxd networking
16:38 <powersj> so no IP is received
16:39 <smoser> ajorg: would you have had outbound access to launchpad https ? if not, then that'd be expected failure.
16:40 <smoser> oh, and i guess 'gh:powersj' (github)
16:40 <ajorg> smoser: I'll check some things, but in short yes. Maybe lxc is being weird?
16:40 <smoser> i dont like our user names in that test though...
16:40 <powersj> smoser: we could use the bot instead
16:42 <ajorg> smoser: it's a public ec2 instance with no special outbound rules, and I can connect to public https sites from a normal session.
16:44 <blackboxsw> hrm, ok let's chat about what we can do to anonymize or drop that type of test data if we can
16:44 <blackboxsw> probably time to kick over to office hours
16:45 <blackboxsw> #topic Office Hours (next 30 minutes)
16:45 <smoser> powersj: well, i think i'd prefer some public key that we state "no one has the private key for this."
16:46 <smoser> obviously we could lie about that, but one would *expect* that you and I would gain access to the system using our public keys.
16:46 <smoser> it doens't make me feel a lot better that a bot could/can.
16:47 <ajorg> Is there a way to limit integration testing to a specific test?
16:47 <blackboxsw> Feel free to bring up any topic/bugs/branches/features you'd like discussion on. We can also continue our discussion on the ssh key imports in teting
16:47 <ajorg> (takes a long time to run the full suite)
16:48 <blackboxsw> ajorg: yes
16:48 <blackboxsw> (reverse-i-search)`cloud_t': python3 -m tests.cloud_tests run --os-name=artful --platform=nocloud-kvm --preserve-data --data-dir=../results --verbose -t modules/locale -t modules/set_password
16:48 <ajorg> thanks, that should help
16:48 <blackboxsw> ajorg: you can specify the test names (like modules/set_password) and modules/locale  in this test
16:48 <blackboxsw> yeah those are short ones I frequently test with
16:49 <smoser> http://paste.ubuntu.com/26438334/
16:49 <smoser> that is what i use. and yeah... we've discussed that integration test could be easier to run :)
16:49 <blackboxsw> #link http://paste.ubuntu.com/26438334/
16:49 <blackboxsw> nice 1
16:51 <blackboxsw> smoser: to have a public key we know nobody has a private key for would that mean we'd need a separate github account (or maybe just an additional key associated w/ our bot account in gh
16:51 * blackboxsw checks github for authorizing multiple keys.
16:51 <blackboxsw> hrm, that wouldn't work as we need gh:ubuntu-server-bot   (one key) n/m
16:52 <ajorg> I've got meetings most of today, so I'll have to follow up later. thanks everyone!
16:52 <blackboxsw> thanks ajorg
16:59 <blackboxsw> so, bot account for the time being is better than powersj owning the testing world ;)
16:59 <blackboxsw> but I'm not too concerned about it as this are supposed to be throw away instances
16:59 <blackboxsw> but I'm not too concerned about it as there instances under test are supposed to be throw away instances
17:00 <blackboxsw> *these instances*.... anyway
17:04 <smoser> blackboxsw: right. it would require users on both those services .
17:14 <blackboxsw> alrighty. think we're at the close of office hours. Last call?
17:16 <blackboxsw> Thanks for your time and contributions to cloud-init folks!
17:16 <blackboxsw> #endmeeting