== Meeting information == * #cloud-init: Cloud-init bi-weekly status meeting, 22 Jan at 16:08 — 17:16 UTC * Full logs at [[http://ubottu.com/meetingology/logs/cloud-init/2018/cloud-init.2018-01-22-16.08.log.html]] == Meeting summary == === Recent changes === The discussion about "Recent changes" started at 16:10. * ''LINK:'' https://github.com/boto/botocore/issues/1351 * ''LINK:'' https://github.com/boto/botocore/issues/1351 * ''LINK:'' https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin === In-progress Development === The discussion about "In-progress Development" started at 16:24. * ''LINK:'' https://code.launchpad.net/~dojordan/cloud-init/+git/cloud-init/+merge/334341 * ''LINK:'' https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-ci-nightly/ === Office Hours (next 30 minutes) === The discussion about "Office Hours (next 30 minutes)" started at 16:45. * ''LINK:'' http://paste.ubuntu.com/26438334/ * ''LINK:'' http://paste.ubuntu.com/26438334/ == Vote results == == Done items == * (none) == People present (lines said) == * blackboxsw (83) * smoser (23) * ajorg (16) * powersj (10) * ubot5 (6) * meetingology (3) * rharper (1) == Full Log == 16:08 #startmeeting Cloud-init bi-weekly status meeting 16:08 Meeting started Mon Jan 22 16:08:22 2018 UTC. The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology. 16:08 16:08 Available commands: action commands idea info link nick 16:08 certianly ajorg :) (on office hours) 16:09 Welcome to another episode of cloud-init bi-weekly status. We'll chat about about cloud-init updates and in progress work, and we'l drop into office hours for ongoing discussions/bug work etc. 16:10 #topic Recent changes 16:11 Just walking through git-log for what we have committed in the last couple of weeks, here's the brief summary 16:12 thx smoser 16:12 - shorten the message in the exception per powersj feedback 16:12 - Use the same botocore session so the patched changes stick. 16:12 - fix bad use of % 16:12 - Fix console_log, improve comments and raise PlatformError on. 16:12 - tests: Fix EC2 Platform to return console output as bytes. 16:12 - tests: remove zesty as supported OS to test [Joshua Powers] 16:12 - Do not log warning on config files that represent None. (LP: #1742479) 16:12 Launchpad bug 1742479 in cloud-init (Ubuntu) "setting manual_cache_clean causes warning" [Medium,Fix released] https://launchpad.net/bugs/1742479 16:12 - tests: Use git hash pip dependency format for pylxd. 16:12 - tests: add integration requirements text file [Joshua Powers] 16:12 - MAAS: add check_instance_id based off oauth tokens. (LP: #1712680) 16:12 - tests: update apt sources list test [Joshua Powers] 16:12 - tests: clean up image properties [Joshua Powers] 16:12 - tests: rename test ssh keys to avoid appearance of leaking private keys. 16:12 Launchpad bug 1712680 in maas-images "cloud-init re-generates network config every reboot overwriting manual admin changes on CentOS." [Undecided,New] https://launchpad.net/bugs/1712680 16:12 [Joshua Powers] 16:12 - tests: Enable AWS EC2 Integration Testing [Joshua Powers] 16:12 - cli: cloud-init clean handles symlinks (LP: #1741093) 16:12 Launchpad bug 1741093 in cloud-init "cloud-init clean traceback on instance dir symlink" [Low,Fix committed] https://launchpad.net/bugs/1741093 16:13 What's being patched in botocore? 16:13 So a number of changes went into integration test related work, separating out requirements files. 16:14 MAASDatasource now also has smarted cache handling based on oauth token renewal from the maas server 16:14 so botocore is used by integration tests only as a mechanism to talk to the instance under test... looking back at the specifics here 16:14 it might have just been shuffling out how and where we define the dependency 16:14 blackboxsw: (my 'paste' to you was bad... http://paste.ubuntu.com/26438113/ is better, showing only those on master, not my local branch that was currently checked out ) 16:15 heh, oopsie daisy let's paste again inline then 16:15 - tests: remove zesty as supported OS to test [Joshua Powers] 16:15 - Do not log warning on config files that represent None. (LP: #1742479) 16:15 - tests: Use git hash pip dependency format for pylxd. 16:15 - tests: add integration requirements text file [Joshua Powers] 16:15 - MAAS: add check_instance_id based off oauth tokens. (LP: #1712680) 16:15 - tests: update apt sources list test [Joshua Powers] 16:15 - tests: clean up image properties [Joshua Powers] 16:15 - tests: rename test ssh keys to avoid appearance of leaking private keys. 16:15 [Joshua Powers] 16:15 - tests: Enable AWS EC2 Integration Testing [Joshua Powers] 16:15 - cli: cloud-init clean handles symlinks (LP: #1741093) 16:15 Launchpad bug 1742479 in cloud-init (Ubuntu) "setting manual_cache_clean causes warning" [Medium,Fix released] https://launchpad.net/bugs/1742479 16:15 Launchpad bug 1712680 in maas-images "cloud-init re-generates network config every reboot overwriting manual admin changes on CentOS." [Undecided,New] https://launchpad.net/bugs/1712680 16:15 Launchpad bug 1741093 in cloud-init "cloud-init clean traceback on instance dir symlink" [Low,Fix committed] https://launchpad.net/bugs/1741093 16:15 ok the real deal, that looks better 16:16 ahh ajorg that interim commit message on botocore was about integration tests caching the session information during testing so we don't recreate that session with every ssh connection to the instance 16:16 just a little time savings per review comments on powersj branch I believe 16:17 okay, so nothing that needs to get upstreamed to botocore? 16:17 I don't think so, powersj smoser I have vague recollection of someone filing an upstream botocore issue. did we have to do that for something else though? 16:18 https://github.com/boto/botocore/issues/1351 16:18 that was the issue smoser put in ^ 16:18 nice recall powersj thanks. 16:18 #link https://github.com/boto/botocore/issues/1351 16:20 ajorg: you can read that bug. imo they have a data loss error, but not one that they can easily fix without causing failures in places that previously ran fine. 16:20 I'll ask them to re-open it. 16:21 At the very least they should answer your last. 16:21 thanks. 16:22 Generally anything significant that we have landed (and any inprogress work) should be available at the following link. 16:22 #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin 16:23 anything else we should note over the last couple weeks? 16:23 otherwise I'll switch to ongoing work topic 16:24 #topic In-progress Development 16:25 As you may have seen last week, we've gotten through a few passes and discussions around dojordan's branch to define pre-provisioning 16:25 #link https://code.launchpad.net/~dojordan/cloud-init/+git/cloud-init/+merge/334341 16:26 some of that discussion resulted in a new context manager: EphemeralDHCPv4 to support a sandboxed dhclient request on an instance. 16:27 this context manager affects Ec2 datasource a bit as it encapsulates all of the dhcp request -> EphemeralIPV4Network calls that Ec2 was doing 16:28 there may be a couple other datasources that follow suit with this type of sandboxed dhcp request in weeks to come 16:28 glad it turned out to be generally useful rather than only specifically to ec2 16:28 absolutely 16:30 Some other in-progress bits look like we might try focusing a bit more on chrony support and gettting robjo's branches some more eyes. 16:31 and some work on Ubuntu snappy support per the snappy and snap config modules. 16:31 dojordan: i just put one comment on your mp. /me thanks dojordan again for his patience. 16:32 rharper: smoser powersj anything more in the immediate pipeline that I'm missing/ 16:32 ? 16:32 blackboxsw: we should get the EphemeralDHCP thingy into the digital ocean datasource also. 16:32 blackboxsw: a reply to the network discussion on the list from the azure folks and robjo 16:32 I took another look at https://code.launchpad.net/~yeazelm/cloud-init/+git/cloud-init/+merge/331897 and saw that origin/master seems to be failing some of the integration tests too. 16:32 (at least for me, locally, on a 16.04 instance) 16:32 ahhh right forgot about all your work there rharper, thanks! 16:33 ajorg: https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-ci-nightly/ 16:33 that is nigytly run of trunk 16:33 #link https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-ci-nightly/ 16:34 I'll try blackholing IMDS on my instance. Could be that's interfering with something. 16:35 it is red, but 218 (green) and 219 (red) used the same git has on trunk (5cc0b19b8). 16:35 I'll follow up during office hours 16:36 can you give me example of your failures ? we had "disk full" errors recently on our jenkins, so that might be the cause of the issue for 291. 16:36 s/291/219/ 16:36 I don't remember seeing that traceback recently. w/ warning messages present in cloud-init 16:36 powersj: ? can you explain lxc timeout failure at 16:36 https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-ci-nightly/219/consoleFull 16:37 smoser: we discovered that our qemu-migration test was installing lxd from the archive and causing conflicts with the snap installed lxd 16:37 I have a message to christian to prevent it, and I have already cleaned it up 16:37 so new runs should pass 16:37 2018-01-22 16:19:03,550 - tests.cloud_tests - WARNING - test case: modules/ssh_import_id failed TestSshImportId.test_no_stages_errors with: AssertionError: 1 != 0 : errors ['(\'ssh-import-id\', ProcessExecutionError("Unexpected error while running command.\\nCommand: [\'sudo\', \'-Hu\', \'ubuntu\', \'ssh-import-id\', \'gh:powersj\', \'lp:smoser\']\\nExit code: 1\\nReason: -\\nStdout: -\\nStderr: -",))'] were encountered in stage m 16:38 hm.. well, that will hit launchpad.net over https 16:38 cloud-init-output.log probaly has more info (should be collected) 16:38 the actual error is: File "/var/lib/jenkins/slaves/torkoal/workspace/cloud-init-ci-nightly/tests/cloud_tests/platforms/instances.py", line 142, in _wait_for_system 16:38 raise OSError('timeout: after {}s system not started'.format(time)) 16:38 it is because when the qemu tests installed lxd it didn't initialize lxd networking 16:38 so no IP is received 16:39 ajorg: would you have had outbound access to launchpad https ? if not, then that'd be expected failure. 16:40 oh, and i guess 'gh:powersj' (github) 16:40 smoser: I'll check some things, but in short yes. Maybe lxc is being weird? 16:40 i dont like our user names in that test though... 16:40 smoser: we could use the bot instead 16:42 smoser: it's a public ec2 instance with no special outbound rules, and I can connect to public https sites from a normal session. 16:44 hrm, ok let's chat about what we can do to anonymize or drop that type of test data if we can 16:44 probably time to kick over to office hours 16:45 #topic Office Hours (next 30 minutes) 16:45 powersj: well, i think i'd prefer some public key that we state "no one has the private key for this." 16:46 obviously we could lie about that, but one would *expect* that you and I would gain access to the system using our public keys. 16:46 it doens't make me feel a lot better that a bot could/can. 16:47 Is there a way to limit integration testing to a specific test? 16:47 Feel free to bring up any topic/bugs/branches/features you'd like discussion on. We can also continue our discussion on the ssh key imports in teting 16:47 (takes a long time to run the full suite) 16:48 ajorg: yes 16:48 (reverse-i-search)`cloud_t': python3 -m tests.cloud_tests run --os-name=artful --platform=nocloud-kvm --preserve-data --data-dir=../results --verbose -t modules/locale -t modules/set_password 16:48 thanks, that should help 16:48 ajorg: you can specify the test names (like modules/set_password) and modules/locale in this test 16:48 yeah those are short ones I frequently test with 16:49 http://paste.ubuntu.com/26438334/ 16:49 that is what i use. and yeah... we've discussed that integration test could be easier to run :) 16:49 #link http://paste.ubuntu.com/26438334/ 16:49 nice 1 16:51 smoser: to have a public key we know nobody has a private key for would that mean we'd need a separate github account (or maybe just an additional key associated w/ our bot account in gh 16:51 * blackboxsw checks github for authorizing multiple keys. 16:51 hrm, that wouldn't work as we need gh:ubuntu-server-bot (one key) n/m 16:52 I've got meetings most of today, so I'll have to follow up later. thanks everyone! 16:52 thanks ajorg 16:59 so, bot account for the time being is better than powersj owning the testing world ;) 16:59 but I'm not too concerned about it as this are supposed to be throw away instances 16:59 but I'm not too concerned about it as there instances under test are supposed to be throw away instances 17:00 *these instances*.... anyway 17:04 blackboxsw: right. it would require users on both those services . 17:14 alrighty. think we're at the close of office hours. Last call? 17:16 Thanks for your time and contributions to cloud-init folks! 17:16 #endmeeting Generated by MeetBot 0.1.5 (http://wiki.ubuntu.com/meetingology)