17:03:25 <balloons> #startmeeting Ubuntu QA
17:03:25 <meetingology> Meeting started Wed Feb 29 17:03:25 2012 UTC.  The chair is balloons. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
17:03:25 <meetingology> 
17:03:25 <meetingology> Available commands: #accept #accepted #action #agree #agreed #chair #commands #endmeeting #endvote #halp #help #idea #info #link #lurk #meetingname #meetingtopic #nick #progress #rejected #replay #restrictlogs #save #startmeeting #subtopic #topic #unchair #undo #unlurk #vote #voters #votesrequired
17:03:39 <balloons> can we get a roll call of who's around
17:03:54 <phillw> o/
17:05:08 <phillw> could be a short meeting :/
17:05:22 <balloons> heh
17:05:24 <brendand> o/
17:05:32 <josepht> o/
17:06:00 <balloons> alrighty.. well let's dive into the agenda
17:06:12 <balloons> there was one outstanding item, for me to update the wiki
17:06:23 * balloons balloons to update QA Team wiki heading
17:06:31 <balloons> i wanted to add an activities icon
17:06:39 <balloons> I seem to only be successful at breaking the header page
17:07:50 <phillw> cproffit from ubuntu-begginers helped me when I got stuck with the new lubuntu header
17:08:21 <balloons> phillw, awesome, I'll consult him.. expect to see that today after the meeting
17:08:32 <balloons> i picked the <3 icon for activities
17:08:33 <balloons> hehe
17:08:51 <balloons> [TOPIC] Ubuntu Updates
17:09:12 <balloons> jibel, are you around?
17:09:24 <jibel> hey
17:09:41 <balloons> hello :-) care to update us on ubuntu qa activities this week?
17:10:26 <jibel> sure, Precise Beta 1 testing in progress
17:10:46 <jibel> list of images that need testing http://iso.qa.ubuntu.com/qatracker/milestones/208/builds
17:11:18 <jibel> a bunch of bugs has been found http://iso.qa.ubuntu.com/qatracker/reports/defects/opened
17:11:34 <jibel> major bugs with the installer and the live session are:
17:11:43 <jibel> bug 940908
17:11:45 <ubottu> Launchpad bug 940908 in ubiquity (Ubuntu Precise) "Keyboard layout not set on persistent USB image" [High,Confirmed] https://launchpad.net/bugs/940908
17:11:52 <jibel> bug 939450
17:11:53 <ubottu> Launchpad bug 939450 in ubiquity (Ubuntu Precise) "ubiquity crashed with TypeError: argument of type 'NoneType' is not iterable in ubi-partman.py" [High,Triaged] https://launchpad.net/bugs/939450
17:11:59 <jibel> bug 942560
17:12:00 <ubottu> Launchpad bug 942560 in ubiquity (Ubuntu) "keyboard layout screen - Keyboard navigation broken" [High,Triaged] https://launchpad.net/bugs/942560
17:12:11 <brendand> o/
17:12:19 <balloons> yes brendand ?
17:12:23 <jibel> and bug 942030
17:12:23 <brendand> http://iso.qa.ubuntu.com/qatracker/milestones/208/builds/12727/testcases
17:12:24 <ubottu> Launchpad bug 942030 in ubiquity (Ubuntu) "'Take a photo' preview pane not showing live preview" [High,Confirmed] https://launchpad.net/bugs/942030
17:12:34 <brendand> http://launchpad.net/bugs/942573
17:12:35 <ubottu> Launchpad bug 942573 in xubuntu-default-settings (Ubuntu) "Network connections indicator not shown" [Undecided,Fix released]
17:12:45 <brendand> is in xubuntu, but marked against an ubuntu testcase??
17:13:53 <jibel> right, I saw that but didn't talk to primes2h yet
17:15:05 <jibel> on the image testing side, a tester reported a hang on the keyboard layout screen on the second stage of an OEM install
17:15:49 <jibel> I've been unable to reproduce, it be nice if we could have more testers to validate that if it is a local issue or not
17:16:31 <balloons> jibel, can we draft some of this up into a mail for the qa list? I'll also send a tweet on status
17:16:32 <jibel> apart from that, beta 1 should be released tomorrow, that's plenty of hours for more coverage
17:16:42 <jibel> balloons, ok
17:16:50 <balloons> thanks jibel!
17:16:53 <jibel> ..
17:17:15 <balloons> Alright, let's talk about flavors now
17:17:24 <balloons> [TOPIC] Ubuntu Flavor Updates
17:17:31 <balloons> lubuntu first, how's that
17:17:37 <phillw> http://iso.qa.ubuntu.com/qatracker/reports/bugs/938472
17:18:05 <phillw> We're all over it like a rash, this was caused by  it's a change on GTK3 3.2.1 syntax and theme standards (I'm angry because of so many changes, they won't make any standard if it goes that way) and Unico 1.0.1 (some textures changes, too).
17:18:19 <phillw> Which pulled the rug from under lubuntu for a week.
17:19:05 <phillw> a rough patch is hoped for today, and then all the isos will need respinning.
17:19:15 <balloons> yikes
17:19:54 <phillw> IMHO, it was an epic fail via QA for this to happen. We had no warning nor information. Just black screens.
17:21:13 <balloons> alrighty. good luck, I'm looking at the screenshot of the issue
17:21:27 <balloons> thata's no fun.. and very vague as to what could be the issue
17:21:30 <phillw> so, the request is - is there any way to stop this in thre future?
17:21:30 <balloons> glad you isolated it
17:21:56 <phillw> our graphics guy is proud of his work and was gutted over it :(
17:22:05 <balloons> :-( that's no good at all
17:22:53 <balloons> I feel like the future for stopping this is doing more testing of key components before they entire the archive
17:22:59 <phillw> the only other thing to note is the ppc *may* not be working on G3 macs, but until we can actually install it's hard for our few testers to test :(
17:23:20 <balloons> I'll sidebar for a moment to mention this list: https://wiki.ubuntu.com/PrecisePangolin/DesktopCriticalPackages
17:23:27 <brendand> o/
17:23:34 <balloons> sure, go ahead brendand
17:23:45 <brendand> is lubuntu being iso tested?
17:23:49 <brendand> in jenkins?
17:24:29 <balloons> I'm not sure phillw ? jibel?
17:25:06 <phillw> balloons: I know very little about jenkins
17:26:34 <jibel> balloons, we test ubuntu, but the code we use for testing is available if you want to run it on your own infrastructure
17:26:35 <jibel> https://code.launchpad.net/~ubuntu-server-iso-testing-dev/ubuntu-server-iso-testing/trunk
17:27:07 <balloons> brendand, do you feel like having jenkins on this would have helped in this case?
17:27:14 <balloons> is that the answer going forward?
17:27:53 <brendand> balloons, well it looks like an integration issue to me
17:28:15 <brendand> balloons, as in Gtk didn't break itself , it broke something else
17:28:45 <balloons> k, let;s move this discussion to the mailing list.. phillw it looks like there may be a way to improve QA for future
17:28:57 <phillw> thanks.
17:29:04 <balloons> i'm sure brendand and jibel will speak more about it there
17:29:05 <jibel> balloons, in this case, for ubuntu, we don't have tests that cover the rendering of the UI and similar defects are found with manual testing
17:29:30 <balloons> jibel, thanks for the info. it certainly depends on what your coverage is
17:29:44 <balloons> ok, how about kubuntu?
17:29:46 <jibel> but the plan is to be able to do this type of test with the next testing framework we are working on
17:29:53 <brendand> phillw - i have an idea to share. on the mailing list - if you can send a message to get things rolling
17:30:20 <Riddell> brendand: looking good for beta I think
17:30:34 <Riddell> we have kubuntu-active images yet but nobody has tested and almost certainly they don't do anything
17:32:07 <balloons> yes.. any thoughts on getting hardware to try those out? will they be a published image for 12.04, or just ?
17:35:34 <balloons> Riddell, thoughts on what will be happening with the kubuntu-active images?
17:36:07 <Riddell> balloons: it'll be a "tech preview" at best
17:36:16 <balloons> ok, makes sens
17:36:19 <Riddell> if we get it going that's nice, if not then we won't worry too much
17:36:31 <balloons> awesome.. anything else?
17:37:12 <balloons> alrighty, let's move on to ubuntu studio.. anyone about care to update us?
17:38:53 <balloons> no one about.. :-)
17:39:01 <balloons> how about edubuntu?
17:40:03 <balloons> lastly, any updates from xubuntu?
17:41:39 <balloons> heh.. kk.. Let's move on
17:41:45 <balloons> we've got alot to chat about
17:42:00 <balloons> [TOPIC] Other topics
17:42:07 <brendand> o/
17:42:12 <balloons> yes brendand ?
17:42:15 <brendand> ok
17:42:17 <brendand> manual testing
17:42:27 <brendand> balloons, love what you're doing with checkbox
17:42:45 <brendand> however
17:43:00 <brendand> the whole story around creating test cases is really messy
17:43:12 <brendand> we have a wiki, but it's not complete
17:43:21 <brendand> we have a spreadsheet, i'm not sure if that's complete
17:43:28 <brendand> we're looking at case conductor
17:43:41 <brendand> and now you can also submit jobs to checkbox
17:44:00 <brendand> (the checkbox-app-testing branch to be precise)
17:44:25 <balloons> brendand, yep your correct on all those things
17:44:33 <brendand> can the wiki die anytime soon?
17:44:38 <balloons> the wiki is not the long-term solution
17:44:55 <balloons> that was discussed last year afaik, and the decision to leave it was made then
17:45:17 <balloons> if not I would propose we go ahead and formally say so now ;-)
17:45:33 <brendand> balloons, yes but then as far as i can tell you're using the test cases as written on the wiki for the checkbox tests...
17:45:52 <brendand> am i wrong?
17:46:02 <balloons> I pulled the test cases we re-wrote as part of the spreadsheet rewrite project
17:46:05 <balloons> that is correct
17:46:19 <brendand> so they are *all* in the spreadsheet?
17:46:26 <balloons> all the checkbox testcases?
17:46:28 <balloons> not all all
17:46:33 <balloons> *not at all
17:46:34 <brendand> ok
17:47:05 <balloons> if you look the wiki has only maybe the nautilus and network manager tests copied unedited
17:47:23 <balloons> i took ideas from some of the others, but by and large, the wiki is pretty old and dead
17:47:29 <brendand> can we make the wiki read-only?
17:47:40 <brendand> (maybe it is already and i'm being a dum-dum)
17:47:56 <balloons> no, it's still editable.. I'm not sure the wiki can or can't be readonly
17:48:02 <balloons> if it can I would be ok with it
17:48:18 <phillw> it can be tagged as depreciated to encourage people NOT to update it.
17:48:33 <balloons> I would say we are at that point
17:48:47 <balloons> as far as the future, we are still looking for a test case management system
17:49:08 <balloons> in my post I outlined my thoughts about that.. i'll repeat them here, and see what you all think/feel
17:49:20 <brendand> also, banners saying "THIS WIKI IS DEPRECATED - SEE <Google Docs Link>"
17:49:41 <phillw> brendand: we can do that :)
17:49:55 <balloons> basically we need a test case mgmt system,  test case delivery system and a test case reporting system
17:50:20 <brendand> what happened with case conductor?
17:50:33 <balloons> right now, we have the wiki more or less as the mgmt system, nothing (we're trying out checkbox now) as the delivry system, and nothing (we're trying out lp and the results tracker) for the reporting system
17:51:01 <brendand> i'm biased, but checkbox is a good delivery system
17:51:14 <balloons> i have been focusing my efforts recently on getting this checkbox prototype up and running
17:51:22 <phillw> balloons: brendand we can chat on ML about a couple of systems the SII have tested out if it would help?
17:51:24 <brendand> launchpad-results may need an extra layer over it to be generally suitable
17:51:34 <balloons> and the case conductor folks have been working on pushing out the new release
17:51:39 <balloons> but we will prototype that as well
17:51:56 <balloons> phillw, yes that would be great
17:52:11 <brendand> balloons, if we wanted to start using case conductor tomorrow, what would the risks be?
17:52:12 <balloons> at this point I feel like we need a nicely intergrated workflow.. but all 3 needs must be met
17:52:24 <balloons> and I'm not biased as to what tools we use
17:52:41 <balloons> brendand, yes I agree results tracker if adopted would need more added to it
17:53:19 <balloons> brendand, if we started using cc tomorrow, no real risks other than we have to adopt it.. and at the moment we have no way of delivering those tests outside of using the web browser
17:53:27 <balloons> we may or may not like that delivery system
17:53:43 <brendand> but it's still miles ahead of a spreadsheet
17:53:59 <balloons> yes.. it is
17:54:05 <brendand> balloons, have you found out does it have an api for pulling testcases?
17:54:12 <balloons> everything in the spreadsheet is in checkbox tests now though
17:54:22 <balloons> brendand, I'm not sure on the api for pulling tests
17:54:45 <brendand> balloons, you've had meeting with the mozilla guys, right?
17:54:55 <balloons> i wasn't attempting to muddy the waters, but I know having checkbox tests does deprecate the other places we've been putting them
17:55:17 <balloons> brendand, yes I've met with the mozilla guys
17:55:44 <balloons> I intend to revisit case conductor within the next couple weeks
17:55:50 <balloons> probably next week
17:56:02 <brendand> so
17:56:13 <brendand> let's clarify all this
17:56:15 <balloons> they were supposed to have a release around early march
17:56:19 <brendand> i'm a contributor
17:56:37 <brendand> and i'm reading your page, so i go and right a testcase in checkbox and propose a merge
17:57:02 <brendand> the first thing you're going to tell me is 'can you write it up in this spreadsheet too'
17:57:04 <brendand> ?
17:57:18 <brendand> s/right/write/
17:57:34 <balloons> brendand, at this point I make no mention of updating a spreadsheet or wiki (nor have I attempted to myself)
17:57:46 <balloons> aka, the checkbox tests are standalone
17:58:37 <balloons> now, we could decide to deprecate all other tests and just go with the checkbox repo.. however I didn't want to do that, even tho short-term it means we have tests in 2 places (I was assuming the wiki would stay as-is for now as the repository of tests)
17:59:02 <balloons> the long term solution is something like case conductor, which I think we all agree with
17:59:09 <balloons> it's simply what do we do in the shortterm?
17:59:53 <brendand> i think there should be some attempt to keep the authoritative location for test cases and the execution tool in sync
17:59:59 <balloons> alright our time is expired.. I'll end the meeting.. can we continue this discussion in #ubuntu-testing?
18:00:08 <brendand> sure
18:00:14 <balloons> for those folks reading the logs we'll post to the ml a summary
18:00:16 <balloons> kk
18:00:19 <balloons> #endmeeting