== Meeting information ==
 * #cloud-init: cloud-init, 02 Oct at 16:05 — 17:29 UTC
 * Full logs at [[http://ubottu.com/meetingology/logs/cloud-init/2017/cloud-init.2017-10-02-16.05.log.html]]



== Meeting summary ==

=== Recent Changes / Highlights ===
The discussion about "Recent Changes / Highlights" started at 16:06.

  * ''LINK:'' https://lists.launchpad.net/cloud-init/msg00106.html 
  * ''LINK:'' https://launchpad.net/cloud-init/+milestone/17.2 
  * please target bugs to the next release

=== In Progress Development / Highlights ===
The discussion about "In Progress Development / Highlights" started at 16:09.

  * ''LINK:'' http://bit.ly/ci-reviews 
  * ''LINK:'' ? 
  * ''LINK:'' https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin 
  * ''LINK:'' https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+ref/taskLimit that's the branch for the task limit
  * ''LINK:'' https://bugs.launchpad.net/cloud-init 
  * ''LINK:'' http://bit.ly/ci-reviews 

=== Open Discussion / Office Hours ===
The discussion about "Open Discussion / Office Hours" started at 16:17.

  * ''LINK:'' http://paste.ubuntu.com/25661692/ 
  * ''LINK:'' http://paste.ubuntu.com/25661714/ 
  * ''LINK:'' https://bugs.launchpad.net/cloud-init/+bug/1720841 
  * ''LINK:'' https://bugs.launchpad.net/cloud-init/+bug/1720841 
  * ''LINK:'' https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/330525 ;-)



== Vote results ==




== Done items ==

 * (none)



== People present (lines said) ==

 * smoser (88)
 * ajorg (51)
 * rharper (28)
 * blackboxsw (25)
 * robjo (4)
 * ubot5 (3)
 * meetingology (3)



== Full Log ==


 16:05 <smoser> #startmeeting cloud-init

 16:05 <meetingology> Meeting started Mon Oct  2 16:05:47 2017 UTC.  The chair is smoser. Information about MeetBot at http://wiki.ubuntu.com/meetingology.

 16:05 <meetingology> 

 16:05 <meetingology> Available commands: action commands idea info link nick

 16:06 <smoser> #topic Recent Changes / Highlights

 16:06 <smoser> hm..

 16:06 <smoser> 17.1 released https://lists.launchpad.net/cloud-init/msg00106.html

 16:06 <smoser> thats the biggest thing and only thing i have for this topic

 16:06 <blackboxsw> #link  https://lists.launchpad.net/cloud-init/msg00106.html

 16:06 <smoser> horay for a release, thanks to those who contributed.

 16:07 <rharper> \o/

 16:07 <smoser> 17.2 is set for 2017-12-14

 16:08 <smoser> https://launchpad.net/cloud-init/+milestone/17.2

 16:08 <ajorg> yay!

 16:08 <smoser> feel free to target bugs to that release.

 16:08 <rharper> #link https://launchpad.net/cloud-init/+milestone/17.2

 16:08 <rharper> #info please target bugs to the next release

 16:09 <smoser> does that do anyting ?

 16:09 <rharper> in the meeting summary, it does

 16:09 <blackboxsw> smoser: the links showed up in meeting minutes last time

 16:09 <smoser> k. i always expected the bot to tell me that in a pm

 16:09 <blackboxsw> meetingology didn't echo though

 16:09 <rharper> well, smoser may have to do those

 16:09 <smoser> #topic In Progress Development / Highlights

 16:09 <blackboxsw> same. but something is up that needs attention.

 16:10 <blackboxsw> I'll properly handle it when publishing

 16:10 <rharper> cool

 16:11 <smoser> Merge Proposals

 16:11 <smoser> #link http://bit.ly/ci-reviews

 16:11 <smoser> there are some there for sure. i know that robjo has some he's interested in, and i think ajorg's instance-identity deserves a look

 16:12 <smoser> as well as simpletable for 17.2

 16:12 <ajorg> I'd be grateful for both, yes.

 16:12 <smoser> i am looking at the 'networkd' one which is a blocker as currently Ubuntu does not work properly on azure

 16:12 <blackboxsw> we should have more bandwidth this week

 16:12 <smoser> and all sysstemd-networkd systems will not work properly on CloudStack.

 16:12 <smoser> anything else there?

 16:13 <robjo> The task limit increase has been tested and solves our problem, thus merging would be great

 16:13 <smoser> robjo, yeah. athat does seem unlikely to cause issue

 16:13 <smoser> Trello Board

 16:13 <smoser> # https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin

 16:13 <rharper> #link  ?

 16:13 <smoser> #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin

 16:14 <robjo> https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+ref/taskLimit that's the branch for the task limit

 16:15 <robjo> Sorry, I can never find the link for the open merge proposals :( problem exists between the keyboard and the chair

 16:16 <blackboxsw> will get eyes on that today robjo as your addZypper is about wrapped

 16:16 <smoser> Bugs

 16:16 <smoser> #link https://bugs.launchpad.net/cloud-init

 16:16 <smoser> just mostly pointers... above.

 16:16 <smoser> for Ubuntu in 16.04, there is one SRU in progress that blackboxsw and i need to verify today

 16:16 <smoser> and then we will be soon looking at doing an SRU of 17.1-ish

 16:17 <robjo> Thanks, still in europe, my day is about done, will pick it up tomorrow

 16:17 <smoser> robjo, link is in topic

 16:17 <smoser> http://bit.ly/ci-reviews

 16:17 <smoser> ok. so open discussion i guess. no w?

 16:17 <smoser> #topic Open Discussion / Office Hours

 16:17 <smoser> we'll hang around and pay attention to pings and discussion fro the next 30  minutes or so.

 16:18 <smoser> and after that, feel free to ping.

 16:19 <ajorg> I'm writing a unit test for a bugfix I'm preparing to submit. It works, but it's slow because readurl retries and waits a second each time

 16:20 <ajorg> I'm a bit of a mock / patch newb. Anyone can help me see how to patch that argument?

 16:20 <smoser> you can feel free to  mock readurl

 16:20 <smoser> and set its return_value and look at calls

 16:21 <smoser> then you dont have to httppretty.

 16:21 <ajorg> The only thing I want to change about it is the sleep though. Is there a straightforward way to just change one arg?

 16:21 <blackboxsw> tests/unittests/test_handler/test_handler_chef.py is an example and tests/unittests/test_datasource/test_maas.py

 16:21 <blackboxsw> I think

 16:21 <ajorg> Hmm, I could do that, yeah. Would be sufficient for this.

 16:22 <ajorg> I'd still like to know if there's an easy way to just change one argument to something that gets called somewhere else.

 16:22 <smoser> you could also mock the time.sleep from url_helper

 16:22 <ajorg> hahaha, yes. that's what I need to do.

 16:22 <rharper> ajorg: if you want to mock the sleep you can decorate the test_ method with @mock.patch(time.sleep)

 16:22 <smoser> we're suggesting other ways generally because there isn't :) at least that i know of.

 16:23 <ajorg> that's cool, patching time.sleep will do nicely

 16:24 <ajorg> is there an integration test that looks at what lands on the console?

 16:24 <ajorg> (going back to my question from before the meeting)

 16:25 <smoser> no. there coudl be on the nocloud-kvm backend

 16:25 <smoser> but i think there is no console access currently on lxd

 16:25 <smoser> and i think that we do not collect console access on nocloud-kvm

 16:25 <smoser> but as you suggest we should for sure

 16:25 <ajorg> k

 16:26 <ajorg> I wondered if systemd might be swallowing my output, or maybe python (boothook in this case is a python script) is making a strange choice when it sets up logging.

 16:27 <rharper> certainly possible;  I know there were issues with cloud-init starting before say rsyslog on non-systemd boots

 16:28 <rharper> and the python logging has changed w.r.t the default configuration;  cloud-init main sort of expects this transition as it starts up in init-local and it has not yet read the cloud-config for logging configuration yet, so it reads that and then does some replay

 16:28 <ajorg> interesting

 16:29 <smoser> ajorg, you're writing to stdout/err with logging from a boothook ?

 16:29 <ajorg> at least I would have thought cloud-init-output.log would contain my logs though, since it's more a redirect of stderr and stdout, right?

 16:29 <ajorg> yeah

 16:29 <rharper> correct

 16:29 <rharper> this is from a bootcmd ?

 16:29 <ajorg> no, #cloud-boothook

 16:29 * rharper hasn't used boothook

 16:29 <smoser> hm..

 16:30 <ajorg> boothooks are super useful

 16:32 <rharper> smoser: when do boot_hooks run ?  local ?  net ?

 16:33 <rharper> I'm not sure yet where stdout/err for boot hooks occur, but you can dump each from the units via:  journalctl -u cloud-init-local.service   (or cloud-init.service)

 16:33 <blackboxsw> "This is the earliest hook available. Note, that there is no mechanism provided for running only once"

 16:34 <blackboxsw> I'm checking the code now (local/pre-local maybe)?

 16:34 <rharper> yeah, saw that;  it runs in-image scripts programs

 16:34 <ajorg> I did that too, and don't see anything.

 16:34 <ajorg> (journalctl)

 16:34 <rharper> you should see some output

 16:34 <rharper> but not seeing your hook in there ?

 16:34 <ajorg> I can tell by other means that the hook ran

 16:34 <ajorg> just don't see that it printed anything

 16:35 <ajorg> my prime suspect if systemd can't do something bad here is that python makes some decision about not actually logging.

 16:35 <ajorg> and I need to be more explicit that it should log

 16:36 <rharper> so, the boot_hook will sub out each part, I see no capture on the stderr/stdout ; so I would expect those to go to whatever is currently capturing those;

 16:37 <rharper> s/sub/subprocess

 16:38 <ajorg> yup, that's what I expect too, and what I recall seeing on Amazon Linux

 16:38 <rharper> since it's using cloud-init's util.subp, there *should* be a debug level message saying 'Running command %s with allowed return codes %s' which maches a path to the boothook

 16:38 <rharper> I would expect to see that in the cloud-init.log

 16:39 <smoser> http://paste.ubuntu.com/25661692/

 16:39 <ajorg> yup, I do see that

 16:39 <rharper> the output of script, I *think* should go to cloud-init-output.log

 16:39 <smoser> i'm testing ^ on serverstack now.

 16:39 <smoser> i verified the commands run on lxd but not look at console there.

 16:39 <ajorg> thanks

 16:40 <smoser> actually.. yeah, BOOTHOOK does run, but doesnt seem to have stdout tied to same place as bootcmd

 16:41 <ajorg> oh?

 16:41 <ajorg> I'm slightly surprised it's not just me

 16:42 <smoser> http://paste.ubuntu.com/25661714/

 16:42 <rharper> it really should be in the journal/console if stdout is not redirected to a file, cloud-init-local service

 16:43 <smoser> that reproduces in lxc

 16:44 <rharper> smoser: in your  instance on serverstack, do you see your BOOTHOOK in journalctl -b -u cloud-init-local.service ?

 16:44 <smoser> ajorg, please go ahead and open a bug.

 16:44 <ajorg> cool (not cool :-P)

 16:44 <ajorg> will do

 16:44 <smoser> # journalctl -b -u cloud-init.service | grep BOOT

 16:44 <smoser> Oct 02 16:37:41 a5 cloud-init[121]: BOOTCMD: Mon, 02 Oct 2017 16:37:41 +0000: a5

 16:44 <smoser> ^ that is from lxd

 16:45 <rharper> and what about the welcome message from cloud-init ?

 16:45 <smoser> i think not.

 16:45 <rharper> bootcmd does:  util.subp(cmd, env=env, capture=False) ; where boot_hook does: util.subp([filepath], env=env)

 16:46 <smoser> yeah, so boot_hook definitely swallowed. and should not be.

 16:47 <rharper> yeah

 16:49 <ajorg> https://bugs.launchpad.net/cloud-init/+bug/1720841

 16:49 <ubot5> Ubuntu bug 1720841 in cloud-init "Output from boothook is not logged" [Undecided,New]

 16:50 <ajorg> Meanwhile I need to setup more explicit logging from that boothook anyway, so I'm okay.

 16:51 <blackboxsw> #link https://bugs.launchpad.net/cloud-init/+bug/1720841

 16:51 <ubot5> Ubuntu bug 1720841 in cloud-init "Output from boothook is not logged" [Medium,Confirmed]

 16:55 <ajorg> Here's another one, which I've confirmed with a unit test and proposed a patch: https://bugs.launchpad.net/cloud-init/+bug/1720844

 16:55 <ubot5> Ubuntu bug 1720844 in cloud-init "UrlError from #include aborts stage" [Undecided,New]

 16:55 <ajorg> (what is this #link thing you're doing there?)

 16:55 <smoser> well, the bot is supposed to care and do somethign with it.

 16:56 <smoser> but /me thinks it doesnt work right :)

 16:56 <blackboxsw> ajorg: since meetingology is still active, the theory is it is keeping track of links  during this meeting

 16:56 <smoser> (during the meeting)

 16:56 <ajorg> ah, okay

 16:57 <ajorg> I think simpletable is completely ready to merge, btw. Any objections? https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/330525

 16:57 <blackboxsw> The theory is it writes the meeting notes out for us so we can publish to https://cloud-init.github.io

 16:57 <ajorg> #link https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/330525 ;-)

 16:57 <blackboxsw> geg

 16:57 <blackboxsw> heh

 16:58 <blackboxsw> ajorg: I think we were good on that changeset, and we only wanted to wait post 17.1 cut to avoid potential regression

 16:59 <ajorg> great

 16:59 <blackboxsw> I'm +1 on that will give it a spin today and then I think we can land it

 17:01 <smoser> blackboxsw, ajorg my thoughts on the simpletable...

 17:01 <ajorg> here it comes...

 17:01 <ajorg> :-P

 17:01 <blackboxsw> :)

 17:02 <smoser> i would like to have a machine friendly output available

 17:02 <smoser> human friendly is good, but machine friendly solves the actual goal of writing the stuff.

 17:02 <ajorg> So I modified it to display *exactly* as prettytable did

 17:02 <smoser> oh really.

 17:02 <smoser> wow

 17:02 <ajorg> which is admittedly not machine friendly

 17:02 <ajorg> not especially

 17:02 <smoser> then i guess i can't object at all.

 17:02 <ajorg> but it's at least as good as prettytable

 17:03 <smoser> in that its backwards compat

 17:03 <smoser> right.

 17:03 <ajorg> ^ this was my goal, to get you to not object

 17:03 <smoser> i still think we should probaly additionally write some machine friendly json

 17:03 <ajorg> agreed, that would be better

 17:03 <smoser> i tihnk having something human friendly is good though

 17:03 <smoser> as i know *I* look at that output

 17:04 <ajorg> it meets my goal of not depending on prettytable

 17:04 <smoser> and parsing json would be less nice

 17:04 <smoser> yeah

 17:05 <smoser> so based on your assertion that it outputs the same as pretty table, i have no objections. only future hopes.

 17:05 <smoser> and i do agree dropping pretty table is nice

 17:05 <ajorg> winning

 17:06 <blackboxsw> smoser: yeah I was wondering how we generally expect people/machines to parse cloud-init-output.log. Right now it's kindof hard to do machine parsing of cloud-init-output.log.

 17:07 <smoser> blackboxsw, yeah. cloud-init-output.log is not intended to be machine friendly.

 17:07 <blackboxsw> do we know already of non-human consumers of the formatting in cloud-init.output.log?

 17:07 <smoser> but the console actually has value in having machien friendly things on it.

 17:08 <smoser> my feeling is if you have access to /var/log/cloud-init.log, then you could very easily have written whatever you wanted to another file that was purely machine friendly.

 17:08 <blackboxsw> was wondering where we would intend to dump machine-friendly json

 17:08 <blackboxsw> yeah

 17:08 <smoser> but the console (/dev/ttyS0) boot log is different

 17:08 <smoser> in that it can give you a couple things you'd not find easy access to

 17:09 <smoser> a.) ssh public keys for the system (providing out of band communication of this data)

 17:09 <smoser> b.) network configuration info: providn useful bits of data on how you might get to this system

 17:10 <smoser> systemd makes it "fun" to get that data to the console in a safe way

 17:12 <ajorg> ugh yes. systemd like to stomp all over your consoles.

 17:19 <blackboxsw> ok so did we want to iterate on simpletable to dump json, or just look for cloud-init to write supplementary json files under /run/cloud-init

 17:20 <blackboxsw> .. after we land ajorg's branch

 17:21 <ajorg> I think it's a good idea to dump json fragments to console for some things, but I suspect smoser will still not want to break anyone who might be using the tables, so probably best to land my branch as-is

 17:23 <smoser> yeah, i agree with ajorg. for now, we can just take a replacement that drops prettytable

 17:23 <smoser> there are improvments to be done there, but this is a simple win

 17:29 <smoser> we'll go ahead and end meeting here.

 17:29 <smoser> i'll still be around. thanks all!

 17:29 <smoser> #endmeeting



Generated by MeetBot 0.1.5 (http://wiki.ubuntu.com/meetingology)