16:01 #startmeeting ubuntu-server-team 16:01 Meeting started Tue Apr 25 16:01:19 2017 UTC. The chair is powersj. Information about MeetBot at http://wiki.ubuntu.com/meetingology. 16:01 16:01 Available commands: action commands idea info link nick 16:01 #topic Review ACTION points from previous meeting 16:01 rbasak, powersj, cpaelzer to draft mysql triage response page like nginx has (carried over) 16:01 Continue to carry over? 16:02 carry further 16:02 #action rbasak, powersj, cpaelzer to draft mysql triage response page like nginx has (carried over) 16:02 * meetingology rbasak, powersj, cpaelzer to draft mysql triage response page like nginx has (carried over) 16:02 I actually wanted to write yesterday, but the page is in lockdown 16:02 smoser to write a release notes entry on cloud-init (ds-identify) 16:02 several relogings didn't fix it 16:02 o/ sorry 16:02 #info smoser completed release note entry for cloud-init 16:02 nacc to write a release notes entry on ipv6 netboot 16:03 powersj: carry forward :/ 16:03 ok :) 16:03 #action nacc to write a release notes entry on ipv6 netboot 16:03 * meetingology nacc to write a release notes entry on ipv6 netboot 16:03 nacc to write a server guide entry on ipv6 netboot 16:03 o/ 16:03 cpaelzer: perhaps follow up with IS if it remains stuck? 16:03 powersj: same 16:04 #action nacc to write a server guide entry on ipv6 netboot 16:04 * meetingology nacc to write a server guide entry on ipv6 netboot 16:04 rharper to write a server guide entry on v2 yaml support in cloud-init 16:04 rbasak: is wiki handled via IS - yes if it is still stuck next time I come there I'll do so 16:04 powersj: carry 16:04 #action rharper to write a server guide entry on v2 yaml support in cloud-init 16:04 * meetingology rharper to write a server guide entry on v2 yaml support in cloud-init 16:04 rharper to write a release notes entry on v2 yaml support in cloud-init 16:04 same? 16:04 same 16:04 #action rharper to write a release notes entry on v2 yaml support in cloud-init 16:04 * meetingology rharper to write a release notes entry on v2 yaml support in cloud-init 16:04 #action powersj to update copy and paste commands for IRC meeting to say artful 16:04 * meetingology powersj to update copy and paste commands for IRC meeting to say artful 16:04 I'm taking that one ^ 16:05 when I fix up my network-doc branch and it lands in cloud-init, I can point to the ReadTheDocs entry 16:05 anything else? 16:05 cpaelzer: yes - managed by IS 16:05 #topic Zesty Development 16:05 One last time... anything for zesty? 16:05 powersj: there was a request to update the paste commands for SRU members, I did that last week 16:05 rharper: thx 16:06 closing out zesty in 3...2.... 1... 16:06 #topic Artful Development 16:06 #link https://wiki.ubuntu.com/ArtfulAardvark/ReleaseSchedule 16:06 importer should be running for the merges -- i think i'm going to do a server all run later today (it will take a day or two, i expect) 16:07 #info nacc: importer running for merges, will take day or two 16:07 we should identify any merges that we intentionally want to do late to pick the next versiosn 16:07 how long does it take end to end? 16:07 I'm currently in touch with HW manufacturers to clairfy on qmeu/libvirt 16:07 dpb1: how long does the importer take? 16:07 yes 16:07 but if others have similar constraints we can make them clear on our "merge schedule" 16:08 dpb1: not predictable -- i think smoser ran it last in this way? 16:08 man 16:08 dpb1: i would guess several days to import the entire history for 850 or so packages 16:08 although we've imported 300+ already 16:08 ok, if we don't know its fine, just curious. 16:08 so maybe less now 16:08 k 16:08 dpb1: speed hasn't been a priority :) 16:08 :) 16:09 was it run for zesty in the same way? 16:09 ahasenack: no 16:09 ahasenack: for zesty it was done by hand when someone asked for a srckpkg to be imported 16:09 by myself, rbasak or smoser 16:09 brave new world! 16:10 yeah 16:10 I noticed one bit that's slow that I have code to speed up, FWIW. 16:10 Just not had a need to address it yet. 16:10 rbasak: excellent, yeah, i was going to file a bug to do some profiling etc 16:10 similar to what smoser had done before and looking at the @lru_cache stuff 16:10 nacc: I suspect the second run of it would be faster? 16:11 dpb1: once we run the 'full import' of the archive (or our subset), subsequent runs will only pick up what is new 16:11 k 16:11 do we need an action for cpaelzer to figure timeline of merges? any other merge work needing action items? 16:11 and that is what the cron job does basically 16:11 powersj: i don't think we need an explicit action 16:12 ok :) 16:12 powersj: but we probably do need to coordinate, etc. 16:12 dpb1: we used to to do that via the blueprint 16:12 and you can see the list of merges from z in that bp 16:12 #subtopic blueprints 16:12 so we might want to, at first, at least copy all those smae srcpkgs, unassigned to aa 16:12 #link https://blueprints.launchpad.net/ubuntu/+spec/servercloud-aa-server-core 16:12 powersj: no action needed atm, but I'd say we need to coordinate the release in general 16:13 #info cpaelzer nacc rbasak need to coordinate merges via blueprint 16:13 I'm not set on blueprint, gdoc, ... (millions of options) but some sort of who-does-what-when is needed 16:13 powersj: thanks 16:13 thanks powersj 16:13 nacc: thoughts on using a board like trello instead of a blueprint? we can still create a blueprint and link to it. 16:13 dpb1 likes trello 16:13 s/nacc// 16:13 I actually thought just that dpb1 when I saw the curtin board 16:14 dpb1: you said those are public-read but write onyl to us right? 16:14 it can be public, we can assign owners, invite other people 16:14 yes 16:14 as long as it's public, i don't particularly care 16:14 I used that for DPDK, I think it would be nice for many things except bug linking 16:14 or is that improved now? 16:14 the blueprint sort of sucks 16:14 * dpb1 volunteers for action to create a skeleton 16:14 Is there anything specific that trello will solve for us over blueprints? 16:14 as it doesn't gate itself 16:14 I do think we need to be careful on this. Ubuntu development takes place on Ubuntu. 16:14 I understand how much better trello could be - just wondering how it applies to our specific case. 16:14 so we can both edit at the same time and things get lost 16:15 smoser: +1 16:15 rbasak: saving my eyes is first on the list. 16:15 * nacc would also be fine with a text file :) 16:15 turning away from LP is a bad thing more from the mood it represents than technically 16:15 rbasak: but it's not a hard line for me. 16:15 and moving Ubuntu server development (or portions there-of) off launchpad makes that inconsistent with other Ubuntu developement. 16:15 we're part of a larger thing here and consistency matters. 16:15 If we did blueprints properly, the status page would give us useful reporting. 16:16 are blueprints rotted or is it just ours? 16:16 rbasak: maybe we should do one properly, what did we miss last time to do so? 16:16 do any other groups use them? 16:16 * cpaelzer <- no blueprint expert at all 16:16 cpaelzer: I think there are some pieces to set on the blueprint and then it appears properly on status.qa.ubuntu.com or wherever it was. 16:16 could we group items for months planned or any other timeline association? 16:17 cpaelzer: ack; 16:17 so we see early when things slip? 16:17 blueprints can do milestones, yes. 16:17 And I believe status reports on that. 16:17 dpb1: we've certainly used blueprints for server; but it's mostly been a task tracker (todo, inprogress, done) and less of a planner w.r.t when we expect it to complete (17.05, 17.06, etc) 16:17 I think it's mostly matter of JFDI 16:18 which translates to who volunteers 16:18 it's not a very rich content, you can't add comments to a task for example 16:18 you would use a bug for that 16:18 saying where you are, what problems you encountered, how they were solved, ping someone else, etc 16:18 I'm guessing 16:18 yes 16:18 it's very typical to create a bug with the feature 16:18 ahasenack: actually every item usually carries a bug which can have details 16:18 Can I propose that as rbasak nacc cpaelzer plan merges, they look into using blueprint more effectively? 16:18 cpaelzer: ok 16:19 Sure 16:19 powersj: fair deal 16:19 #info rbasak nacc cpaelzer will look into using blueprints more effectively 16:19 yeah, and let's (probably) target 18.04 cycle for dtrt 16:19 OK, let's try it 16:19 from the get-got that is 16:19 #subtopic Release Bugs 16:19 #link http://reqorts.qa.ubuntu.com/reports/rls-mgr/rls-z-tracking-bug-tasks.html#ubuntu-server 16:19 I don't see a -a version of that page yet 16:20 how does a bug get into that list? 16:20 powersj: i wonder if whoever owns reqorts needs to update their code :) 16:20 do we know who does? 16:20 no idea 16:21 #action powersj to find out who owns bug tracking site, get artful link 16:21 * meetingology powersj to find out who owns bug tracking site, get artful link 16:21 any comments on z bugs before moving on? 16:21 ahasenack: i believe it might be automatic? 16:21 I mean, via a tag? zesty task? 16:22 nacc: based on what 16:22 right 16:22 tag of release 16:22 ahasenack: as in, packages in the list of 'ubuntu-server' (which is a file) and then tag to that release 16:22 *task to that release 16:22 https://bugs.launchpad.net/ubuntu/+source/ebtables/+bug/1645324 is in the list 16:22 Launchpad bug 1645324 in ebtables (Ubuntu Trusty) "ebtables: Lock file handling has races" [Medium,In progress] 16:22 oh it is task? 16:22 has tags "patch" and "sts-sru-needed" 16:22 ah, task 16:22 powersj: the one i randomly looked at had no tags 16:22 nacc: ok thx 16:22 i would need to grep my irc logs to find where the package list 16:23 nacc: that's just the package-team-mapping right? 16:23 powersj: that'd be my guess, yes 16:23 http://people.canonical.com/~ubuntu-archive/package-team-mapping.json 16:23 rharpers json? 16:23 powersj: thanks, that's what i was looking for 16:23 so these are essentially sru candidates 16:23 at least they were nominated 16:23 ahasenack: or active development candidates 16:23 ahasenack: now they are SRU, yes ;0 16:23 :) 16:24 which brings us to.... 16:24 #topic Server & Cloud Bugs & SRU/Pending Uploads (slashd) 16:24 #link http://reqorts.qa.ubuntu.com/reports/rls-mgr/rls-x-tracking-bug-tasks.html#ubuntu-server 16:24 welcome slashd 16:24 thanks powersj 16:24 powersj: thanks for driving this to not stall - we are all talkative today 16:24 LP: #1356211 - (sosreport) cannot collect rotated syslog.1 16:24 - Series to SRU : trusty 16:24 - Owners : Louis Bouchard 16:24 LP: #1366103 - (sosreport) sosreport doesn't collect all data from /etc/apparmor.d/ 16:24 - Series to SRU : trusty 16:24 :) 16:24 Launchpad bug 1356211 in sosreport (Ubuntu Trusty) "cannot collect rotated syslog.1" [Medium,In progress] https://launchpad.net/bugs/1356211 16:24 - Owners : Louis Bouchard 16:24 Launchpad bug 1366103 in sosreport (Ubuntu Trusty) "sosreport doesn't collect all data from /etc/apparmor.d/" [Medium,In progress] https://launchpad.net/bugs/1366103 16:24 LP: #1447695 - (sosreport) sosreport 3.2 does not collect systemd information 16:24 Launchpad bug 1447695 in sosreport (Ubuntu Trusty) "sosreport 3.2 does not collect systemd information" [Medium,In progress] https://launchpad.net/bugs/1447695 16:24 - Series to SRU : trusty 16:24 - Owners : Louis Bouchard 16:24 LP: #1566508 - (sssd) autofs races with sssd on startup 16:24 Launchpad bug 1566508 in sssd (Ubuntu Trusty) "autofs races with sssd on startup" [Medium,In progress] https://launchpad.net/bugs/1566508 16:24 - Series to SRU : yakkety xenial trusty 16:24 - Verification : verification-failed-xenial verification-failed-yakkety verification-needed 16:24 - Owners : Victor Tapia 16:24 LP: #1590799 - (nfs-utils) nfs-kernel-server does not start because of dependency failure 16:24 Launchpad bug 1590799 in nfs-utils (Ubuntu Yakkety) "nfs-kernel-server does not start because of dependency failure" [Medium,Fix committed] https://launchpad.net/bugs/1590799 16:24 - Series to SRU : yakkety 16:24 - Verification : verification-done-xenial verification-needed 16:24 - Owners : Rafael David Tinoco 16:25 LP: #1602057 - (nova) [SRU] (libvirt) KeyError updating resources for some node, guest.uuid is not in BDM list 16:25 Launchpad bug 1602057 in nova (Ubuntu Xenial) "[SRU] (libvirt) KeyError updating resources for some node, guest.uuid is not in BDM list" [Medium,Triaged] https://launchpad.net/bugs/1602057 16:25 - Series to SRU : xenial 16:25 - Owners : Edward Hope-Morley 16:25 LP: #1605243 - (sosreport) iscsi initiator related configs and logs are not recorded 16:25 Launchpad bug 1605243 in sosreport (Ubuntu Zesty) "iscsi initiator related configs and logs are not recorded" [Medium,In progress] https://launchpad.net/bugs/1605243 16:25 - Series to SRU : yakkety xenial trusty zesty 16:25 - Owners : Louis Bouchard 16:25 LP: #1614052 - (sosreport) SOSREPORT need to collect OPAL msglog 16:25 Launchpad bug 1614052 in sosreport (Ubuntu Zesty) "SOSREPORT need to collect OPAL msglog" [Medium,In progress] https://launchpad.net/bugs/1614052 16:25 - Series to SRU : yakkety xenial trusty zesty 16:25 - Owners : Louis Bouchard 16:25 LP: #1636322 - (ceph) [SRU] upstart: ceph-all service starts before networks up 16:25 Launchpad bug 1636322 in ceph (Ubuntu Trusty) "[SRU] upstart: ceph-all service starts before networks up" [High,In progress] https://launchpad.net/bugs/1636322 16:25 - Series to SRU : xenial trusty 16:25 - Verification : verification-kilo-needed verification-liberty-needed 16:25 - Owners : James Page None 16:25 soon the server will ban you :-) 16:25 LP: #1645324 - (ebtables) ebtables: Lock file handling has races 16:25 Launchpad bug 1645324 in ebtables (Ubuntu Trusty) "ebtables: Lock file handling has races" [Medium,In progress] https://launchpad.net/bugs/1645324 16:25 - Series to SRU : trusty zesty 16:25 - Owners : Dragan S. 16:25 LP: #1648242 - (neutron) [SRU] Failure to retry update_ha_routers_states 16:25 Launchpad bug 1648242 in neutron (Ubuntu Xenial) "[SRU] Failure to retry update_ha_routers_states" [Low,Triaged] https://launchpad.net/bugs/1648242 16:25 - Series to SRU : xenial 16:25 - Owners : Edward Hope-Morley 16:25 LP: #1654600 - (unattended-upgrades) unattended-upgrade-shutdown hangs when /var is a separate filesystem 16:25 Launchpad bug 1654600 in unattended-upgrades (Ubuntu Zesty) "unattended-upgrade-shutdown hangs when /var is a separate filesystem" [High,In progress] https://launchpad.net/bugs/1654600 16:25 - Series to SRU : xenial yakkety zesty 16:25 - Verification : verification-failed 16:25 - Owners : Louis Bouchard 16:26 LP: #1657489 - (qemu) qemu-xen: free all the pirqs for msi/msix when driver unload 16:26 Launchpad bug 1657489 in qemu (Ubuntu Trusty) "qemu-xen: free all the pirqs for msi/msix when driver unload" [Undecided,New] https://launchpad.net/bugs/1657489 16:26 - Series to SRU : trusty 16:26 - Owners : None 16:26 LP: #1666827 - (horizon) Backport fixes for Rename Network return 403 Error 16:26 Launchpad bug 1666827 in horizon (Ubuntu Xenial) "Backport fixes for Rename Network return 403 Error" [High,Triaged] https://launchpad.net/bugs/1666827 16:26 - Series to SRU : xenial 16:26 - Owners : None 16:26 #info SRU pending for : sssd, nfs-utils, qemu, ebtables, unattended-upgrades, sosreport 16:26 I appreciate the report - but honestly, I'm not reading that flood. I wonder if anyone is? I suggest sticking to just the list of packages. That single last line would do. Anyone who wants more info knows where to look. 16:26 I read it 16:26 here, let me paste it for you again 16:26 there are a lot of sosreports... 16:26 sorry guys, will do a pastebin next week ;) 16:26 * dpb1 slaps ahasenack 16:26 slashd: thx :) 16:26 but I was good wit hthe short list and people polling on details where needed 16:27 #info SRU pending for: sssd, nfs-utils, qemu, ebtables, unattended-upgrades, sosreport 16:27 questions for slashd? 16:27 sssd? hm 16:27 #info further SRU pending openssh (X, Y), sssd (T), qemu (Xx2), libvirt (X,Y,Z) 16:27 I have my own set of shi$ :-) 16:28 ok moving on, thanks slashd 16:28 #topic Weekly Updates & Questions for the QA Team (powersj) 16:28 #link https://jenkins.ubuntu.com/server/ 16:28 and of course I'm not prepared 16:29 This week I'm looking into some additional CI/pkg build for cloud-init and curtin. And getting the automated ISO tests running again for previous releases. 16:29 What have you done with the real powersj? :-P 16:29 haha 16:29 Last week was the ericson qemu/hugepages QA, some SRU verification attempts on openssh and tomcat7/8, and I build my own local MAAS setup 16:29 Oh and I broke curtin vmtest badly 16:30 and won't do that again, I have learned my lesson 16:30 #info powersj to get automated ISO tests running again 16:30 QA Team all about breaking things. 16:30 amazing 16:30 and on testing fyi - others might just as the virt tests run into cases that there are no lxd/cloud images for artful yet 16:30 powersj: lol 16:30 #info won't break curtin vmtest again 16:30 haha 16:30 uh powersj; we will quote that the next time 16:30 I wouldn't have recorded that 16:30 please do :) 16:31 anything else for the qa noob 16:31 #topic Weekly Updates & Questions for the Kernel Team (smb, sforshee) 16:31 Noting to report but will listen to questions/requests if there are any (smb not that much more prepared). 16:31 smoser: did anybody else delete the wrong kernels because of special ordering ber LANG ? 16:32 smb: any complaints about 4.10 getting removed unintended to the kernel team? 16:32 apt-get autoremove deleted wrong kernels? 16:32 na, purge-kernels did 16:32 purge-old-kernels 16:32 what is purge-old-kernels 16:32 cpaelzer, not that we encourage people to use that :) 16:32 hehe, you are safe anyway smb 16:33 Is that the one inside byobu packaging? 16:33 languages that do thousand separator on "." are good :-) 16:33 yes rbasak 16:33 rbasak: o.O 16:33 If autoremove works just as well, should we deprecate and remove that? 16:33 https://bugs.launchpad.net/ubuntu/+source/byobu/+bug/1685338 16:33 Launchpad bug 1685338 in byobu (Ubuntu) "purge-old-kernels sorts kernels incorrectly" [High,Triaged] 16:33 smb: apt-get autoremove is what you all want, right? 16:33 smoser: you are a user of that, please speak for or against the tool in contrast to autoremove 16:34 rbasak, i think the are probably right that we shoudl deprecate or just remove. 16:34 I only happen to know it from a few bugs 16:34 rbasak, I remember some people were afraid of autoremove because it might remover other stuff (though it never did remove anything for me which I still needed) 16:34 i have used since before autoremove dtrt 16:34 * dpb1 votes to remove this script from byobu 16:34 one reason to keep it would be that it specifically removes old kernels 16:34 how do I rogers rule of meeting etiquite do that? 16:35 where as autoremove autoremoves whatever 16:35 did kikland create it, or did he just added something already existing to byobu? 16:35 possibly autoremove takes a filter.. i dont know. 16:35 AIUI, he wrote it, and stuck it in with byobu for lack of somewhere else to put it. 16:35 Let me file a bug at least 16:36 rbasak: for removal? 16:36 https://askubuntu.com/questions/2793/how-do-i-remove-old-kernel-versions-to-clean-up-the-boot-menu 16:36 rbasak, kirkland got something gfrom kees, there was a ubuntu-devel discussion on this i think 16:36 http://blog.dustinkirkland.com/2016/06/purge-old-kernels.html 16:36 oh my goodness. 16:37 ok 16:37 "You'll already have the purge-old-kernels command in Ubuntu 16.04 LTS (and later), as part of the byobu package. " 16:37 The definitive answer is: sudo purge-old-kernels 16:37 so there you have it 16:37 in the case where a user does "someting" and autoremove starts to not dtrt 16:37 purge-old-kernels works 16:38 *but* the real goal is to figure out what "something" was and why autoremove stops dtrt 16:38 (imo) 16:38 and fix it 16:38 honestly though, if it's the definitive answer, then it should be a stand-alone package and seeded 16:38 bug 1686138 16:38 bug 1686138 in byobu (Ubuntu) "purge-old-kernels is superseded by "apt autoremove"" [Undecided,New] https://launchpad.net/bugs/1686138 16:38 no 16:38 oftentimes, i've found, it's pinning from an old release 16:38 rharper: agreed. 16:38 I think we need a follow up 16:38 sudo apt-mark auto `apt-mark showmanual '^linux-(headers|image|image-extra|cloud-tools|tools)-[0-9]'` 16:38 dpb1: ack 16:38 ^ I think that should fix autormeove 16:38 smb: that's user friendly 16:39 rharper, also if it is the definitive answer, it should not remove the wrong kernels 16:39 smoser: ack 16:39 even if you do not speak german 16:39 hehe 16:39 smb: correct, my imporession as well 16:39 smb: the problem is joe user who intentionally pinned an old kernel and then un-marking it is wrong 16:39 anything else on removing old kernels? 16:39 I'll contact dustin 16:40 you can action me 16:40 #action dpb1 to talk to kirkland re: old kernels 16:40 * meetingology dpb1 to talk to kirkland re: old kernels 16:40 anything else for smb 16:40 #topic Upcoming Call For Papers 16:40 #link https://lwn.net/Calendar/Monthly/cfp/ 16:41 I don't see anything 16:42 #topic Ubuntu Server Team Events 16:42 any events? 16:42 tomorrow... tomorrow... 16:42 does coffee and wine count? 16:43 #info Bug squashing day 3 tomorrow! 16:43 #link https://lists.ubuntu.com/archives/ubuntu-server/2017-April/007524.html 16:43 #topic Open Discussion 16:43 Free for all... anything else before closing 16:44 nada... 16:44 #topic Announce next meeting date, time and chair 16:44 same time, same place, with cpaelzer at the helm 16:44 cheers 16:44 #endmeeting