18:01:26 <balloons> #startmeeting ubuntu qa part II 18:01:27 <meetingology> Meeting started Wed Feb 29 18:01:26 2012 UTC. The chair is balloons. Information about MeetBot at http://wiki.ubuntu.com/meetingology. 18:01:27 <meetingology> 18:01:27 <meetingology> Available commands: #accept #accepted #action #agree #agreed #chair #commands #endmeeting #endvote #halp #help #idea #info #link #lurk #meetingname #meetingtopic #nick #progress #rejected #replay #restrictlogs #save #startmeeting #subtopic #topic #unchair #undo #unlurk #vote #voters #votesrequired 18:01:42 <balloons> hehe, it's better than not capturing it, even if there will be noise 18:02:25 <balloons> alright, so yes short term, what do we do.. my thoughts on what to do with the checkbox tests after we had a test management tool was to migrate them at that point 18:02:47 <balloons> and if we still had / used the wiki (or whatever, spreadsheet, whatever) they would also be migrated 18:03:07 <balloons> our future delivery system should simply use the tests from the management system.. so they are maintained in one place 18:03:10 <brendand> balloons, don't you think it would be better if you *just* ask people to add to the spreadsheet and then you and a few other people deal with updating checkbox? 18:03:20 <balloons> and by having a nice test case management system everyone can see and update them easily 18:03:27 <balloons> without having to be a dev, or use bzr 18:04:14 <balloons> brendand, for anyone who can propose a merge, it makes it easier on me to accept the request.. if you can't I am pointing folks to adding the case to the wiki and I will manually import it 18:05:57 <brendand> ok. let's make an action item to clarify that the wiki should not be used except by those who know they need to use it 18:06:02 <balloons> for example, roignac has sent several merge requests that made it really nice on me for just accepting and not having to re-work his test cases 18:06:20 <balloons> brendand, I'm also concerned about the spreadsheet 18:06:37 <balloons> what does everyone think we should do in the short term for holding our test cases 18:06:46 <balloons> put aside the issue of how to contribute for a moment 18:08:15 <brendand> balloons, you're concerned as in you don't like it? 18:08:43 <balloons> brendand, I think I like the spreadsheet even less than the wiki, simply because it's confusing for people 18:08:58 <balloons> and everything in the spreadsheet is supposed to be/ should be in the wiki 18:09:28 <brendand> it's something of a least worst option situation 18:10:48 <balloons> for example, we could deprecate the wiki, and say checkbox branch has our testcases in it. if you want to contribute a new test and can't we could have an open spreadsheet to do so, we would manually import it, then remove the spreadsheet 18:11:03 <balloons> my concerns with that would be making it hard to see all the testcases for everyone 18:11:13 <balloons> and making it hard for the community to edit testcases 18:11:19 <balloons> however it would be a short-term thing 18:11:49 <brendand> right. i think a spreadsheet is better than a wiki for reviewing 18:12:36 <balloons> does anyone have an alternative option to present? 18:12:59 <brendand> i seem to recall that one of the reasons we moved away from the wiki was not because of it being a wiki but because many of the test cases themselves were actually useless 18:13:04 <balloons> I'm not necessarily pushing the option above, just that it was an option 18:13:19 <balloons> brendand, yes the wiki was in worse shape that I thought 18:13:32 <balloons> anything not written recently was old and useless 18:13:37 <balloons> in many cases it was blank 18:14:36 <brendand> i think we need to head toward the optimum solution post-haste 18:15:20 <brendand> make checkbox the definitive store until case-conductor is in place 18:15:50 <brendand> for those who don't want to use bzr we have a 'new test cases' sheet as balloons suggested 18:16:09 <brendand> this is reviewed every QA meeting and synced to checkbox, then wiped 18:16:31 <balloons> if there's no other ideas put forward I'm ok with doing that 18:17:21 <brendand> once case-conductor is in place AND we have a tool which syncs down test cases to checkbox, we can start putting test cases in there 18:17:55 <balloons> sounds fine to me 18:18:21 <balloons> do we feel a spreadsheet is necessary? 18:18:44 <balloons> would the mailing list suffice? or is it ok to simply ask them to bzr branch and propose a merge? 18:19:16 <brendand> balloons, it might limit the scope of contribution, but maybe for the short term 18:19:23 <brendand> ML could work 18:19:36 <brendand> i was just thinking to have an outlet for those with no bzr experience 18:20:20 <brendand> balloons, and there's still the question of the testcases.qa wiki :) 18:20:22 <balloons> yes I'm concerned about that 18:20:27 <balloons> i don't want to limit 18:20:35 <balloons> but perhaps just sending a mail would be best 18:21:34 * phillw has no bzr knowledge :/ 18:21:52 <balloons> awesome.. so phillw what makes sense for you 18:22:05 <balloons> would you feel put off by having the tests in a bzr branch? 18:22:35 <balloons> for instance, you can see all the tests right now by browsing them online 18:22:35 <balloons> http://bazaar.launchpad.net/~nskaggs/checkbox/checkbox-app-testing/files/head:/jobs/ 18:22:46 <phillw> balloons: it is a case of learning and then teaching new testers. My concern is that if it is too complicated it will put them off. 18:23:53 <balloons> phillw, what would you propose then? 18:25:17 <phillw> balloons: the most difficult thing of all. A clear, easy to follow set of instructions for newcomers (And, if I can follow them, any one can.... trust me on that one when I made pcman write up how to set up debug in pcmanfm :P ) 18:25:59 <balloons> phillw, +1. we are working towards that 18:26:21 <balloons> and this decision factors in.. we want this to get iteratively easier, not harder :-) 18:26:32 <phillw> I was also the 'idiot' who idiot checked the GRUB 2 instructions way back when drs305 was writing them :) 18:26:35 <balloons> but the biggest piece behind this checkbox thing was to get more testers out there 18:26:41 <balloons> and I think / hope it will 18:27:21 <balloons> this problem is sort of a side issue.. the main issue I was trying to solve was to get more testing, and better metrics on testing.. in addition, I wanted to increase the number of real tests 18:27:28 <balloons> and get the tests we wrote used 18:30:07 <phillw> I can only say from personal experience, that the likes of http://testcases.qa.ubuntu.com/Plans/LaptopTesting#System are followable? 18:33:42 <brendand> balloons, i need to leave the conversation now. look forward to a few merges into your checkbox branch from me ;) 18:35:25 <balloons> brendand, awesome thanks 18:35:44 <balloons> phillw, are you trying to say you can or can't follow that wiki page? 18:35:56 * balloons is confused on sarcasm or not 18:36:05 <phillw> balloons: I thought the wiki is due to go? 18:36:20 <balloons> phillw, long term yes 18:36:30 <balloons> but short term.. well we're deciding that now :-) 18:37:06 <phillw> balloons: do you have the link to the wiki, please? 18:37:29 <balloons> link to the wiki? this? http://testcases.qa.ubuntu.com/ 18:37:30 <phillw> I'm sufferring from tab overload 18:37:37 <balloons> ahh 18:38:34 <phillw> balloons: that looks familiar aka http://testcases.qa.ubuntu.com/Plans/LaptopTesting#System 18:39:05 <phillw> it is simply 'what do you expect to happen" followed by - Does it? 18:39:15 <balloons> so phillw.. here's a good example.. the nautilus tests are the same in the wiki and checkbox 18:39:16 <balloons> http://testcases.qa.ubuntu.com/Applications/Nautilus 18:39:23 <balloons> http://bazaar.launchpad.net/~nskaggs/checkbox/checkbox-app-testing/view/head:/jobs/nautilus.txt.in 18:39:42 <balloons> phillw, yes the format of those tests are really iffy 18:39:54 <balloons> they would need reworked to go into checkbox or case conductor, etc 18:40:42 <phillw> ooh, pink! Yeah, they are in plain english as to what to expect etc. 18:41:26 <phillw> Sorry, I've GTG - Dinner is served at 18:30 UTC on wednesdays here as I have the lubuntu meeting at 20:00 UTC. 18:42:45 <balloons> gotcha 18:42:47 <balloons> ok, well 18:43:06 <balloons> I'll sumarize on the mailing list 18:43:11 <balloons> thanks for the extended discussion 18:43:18 <balloons> #endmeeting