#title #ubuntu-testing: ubuntu qa part II Meeting started by balloons at 18:01:26 UTC. The full logs are available at http://ubottu.com/meetingology/logs/ubuntu-testing/2012/ubuntu-testing.2012-02-29-18.01.log.html . == Meeting summary == ''LINK:'' http://bazaar.launchpad.net/~nskaggs/checkbox/checkbox-app-testing/files/head:/jobs/ (balloons, 18:22:35) ''LINK:'' http://testcases.qa.ubuntu.com/Applications/Nautilus (balloons, 18:39:16) ''LINK:'' http://bazaar.launchpad.net/~nskaggs/checkbox/checkbox-app-testing/view/head:/jobs/nautilus.txt.in (balloons, 18:39:23) Meeting ended at 18:43:18 UTC. == Votes == == Action items == * (none) == People present (lines said) == * balloons (58) * brendand (16) * phillw (12) * meetingology (3) == Full Log == 18:01:26 #startmeeting ubuntu qa part II 18:01:27 Meeting started Wed Feb 29 18:01:26 2012 UTC. The chair is balloons. Information about MeetBot at http://wiki.ubuntu.com/meetingology. 18:01:27 18:01:27 Available commands: #accept #accepted #action #agree #agreed #chair #commands #endmeeting #endvote #halp #help #idea #info #link #lurk #meetingname #meetingtopic #nick #progress #rejected #replay #restrictlogs #save #startmeeting #subtopic #topic #unchair #undo #unlurk #vote #voters #votesrequired 18:01:42 hehe, it's better than not capturing it, even if there will be noise 18:02:25 alright, so yes short term, what do we do.. my thoughts on what to do with the checkbox tests after we had a test management tool was to migrate them at that point 18:02:47 and if we still had / used the wiki (or whatever, spreadsheet, whatever) they would also be migrated 18:03:07 our future delivery system should simply use the tests from the management system.. so they are maintained in one place 18:03:10 balloons, don't you think it would be better if you *just* ask people to add to the spreadsheet and then you and a few other people deal with updating checkbox? 18:03:20 and by having a nice test case management system everyone can see and update them easily 18:03:27 without having to be a dev, or use bzr 18:04:14 brendand, for anyone who can propose a merge, it makes it easier on me to accept the request.. if you can't I am pointing folks to adding the case to the wiki and I will manually import it 18:05:57 ok. let's make an action item to clarify that the wiki should not be used except by those who know they need to use it 18:06:02 for example, roignac has sent several merge requests that made it really nice on me for just accepting and not having to re-work his test cases 18:06:20 brendand, I'm also concerned about the spreadsheet 18:06:37 what does everyone think we should do in the short term for holding our test cases 18:06:46 put aside the issue of how to contribute for a moment 18:08:15 balloons, you're concerned as in you don't like it? 18:08:43 brendand, I think I like the spreadsheet even less than the wiki, simply because it's confusing for people 18:08:58 and everything in the spreadsheet is supposed to be/ should be in the wiki 18:09:28 it's something of a least worst option situation 18:10:48 for example, we could deprecate the wiki, and say checkbox branch has our testcases in it. if you want to contribute a new test and can't we could have an open spreadsheet to do so, we would manually import it, then remove the spreadsheet 18:11:03 my concerns with that would be making it hard to see all the testcases for everyone 18:11:13 and making it hard for the community to edit testcases 18:11:19 however it would be a short-term thing 18:11:49 right. i think a spreadsheet is better than a wiki for reviewing 18:12:36 does anyone have an alternative option to present? 18:12:59 i seem to recall that one of the reasons we moved away from the wiki was not because of it being a wiki but because many of the test cases themselves were actually useless 18:13:04 I'm not necessarily pushing the option above, just that it was an option 18:13:19 brendand, yes the wiki was in worse shape that I thought 18:13:32 anything not written recently was old and useless 18:13:37 in many cases it was blank 18:14:36 i think we need to head toward the optimum solution post-haste 18:15:20 make checkbox the definitive store until case-conductor is in place 18:15:50 for those who don't want to use bzr we have a 'new test cases' sheet as balloons suggested 18:16:09 this is reviewed every QA meeting and synced to checkbox, then wiped 18:16:31 if there's no other ideas put forward I'm ok with doing that 18:17:21 once case-conductor is in place AND we have a tool which syncs down test cases to checkbox, we can start putting test cases in there 18:17:55 sounds fine to me 18:18:21 do we feel a spreadsheet is necessary? 18:18:44 would the mailing list suffice? or is it ok to simply ask them to bzr branch and propose a merge? 18:19:16 balloons, it might limit the scope of contribution, but maybe for the short term 18:19:23 ML could work 18:19:36 i was just thinking to have an outlet for those with no bzr experience 18:20:20 balloons, and there's still the question of the testcases.qa wiki :) 18:20:22 yes I'm concerned about that 18:20:27 i don't want to limit 18:20:35 but perhaps just sending a mail would be best 18:21:34 * phillw has no bzr knowledge :/ 18:21:52 awesome.. so phillw what makes sense for you 18:22:05 would you feel put off by having the tests in a bzr branch? 18:22:35 for instance, you can see all the tests right now by browsing them online 18:22:35 http://bazaar.launchpad.net/~nskaggs/checkbox/checkbox-app-testing/files/head:/jobs/ 18:22:46 balloons: it is a case of learning and then teaching new testers. My concern is that if it is too complicated it will put them off. 18:23:53 phillw, what would you propose then? 18:25:17 balloons: the most difficult thing of all. A clear, easy to follow set of instructions for newcomers (And, if I can follow them, any one can.... trust me on that one when I made pcman write up how to set up debug in pcmanfm :P ) 18:25:59 phillw, +1. we are working towards that 18:26:21 and this decision factors in.. we want this to get iteratively easier, not harder :-) 18:26:32 I was also the 'idiot' who idiot checked the GRUB 2 instructions way back when drs305 was writing them :) 18:26:35 but the biggest piece behind this checkbox thing was to get more testers out there 18:26:41 and I think / hope it will 18:27:21 this problem is sort of a side issue.. the main issue I was trying to solve was to get more testing, and better metrics on testing.. in addition, I wanted to increase the number of real tests 18:27:28 and get the tests we wrote used 18:30:07 I can only say from personal experience, that the likes of http://testcases.qa.ubuntu.com/Plans/LaptopTesting#System are followable? 18:33:42 balloons, i need to leave the conversation now. look forward to a few merges into your checkbox branch from me ;) 18:35:25 brendand, awesome thanks 18:35:44 phillw, are you trying to say you can or can't follow that wiki page? 18:35:56 * balloons is confused on sarcasm or not 18:36:05 balloons: I thought the wiki is due to go? 18:36:20 phillw, long term yes 18:36:30 but short term.. well we're deciding that now :-) 18:37:06 balloons: do you have the link to the wiki, please? 18:37:29 link to the wiki? this? http://testcases.qa.ubuntu.com/ 18:37:30 I'm sufferring from tab overload 18:37:37 ahh 18:38:34 balloons: that looks familiar aka http://testcases.qa.ubuntu.com/Plans/LaptopTesting#System 18:39:05 it is simply 'what do you expect to happen" followed by - Does it? 18:39:15 so phillw.. here's a good example.. the nautilus tests are the same in the wiki and checkbox 18:39:16 http://testcases.qa.ubuntu.com/Applications/Nautilus 18:39:23 http://bazaar.launchpad.net/~nskaggs/checkbox/checkbox-app-testing/view/head:/jobs/nautilus.txt.in 18:39:42 phillw, yes the format of those tests are really iffy 18:39:54 they would need reworked to go into checkbox or case conductor, etc 18:40:42 ooh, pink! Yeah, they are in plain english as to what to expect etc. 18:41:26 Sorry, I've GTG - Dinner is served at 18:30 UTC on wednesdays here as I have the lubuntu meeting at 20:00 UTC. 18:42:45 gotcha 18:42:47 ok, well 18:43:06 I'll sumarize on the mailing list 18:43:11 thanks for the extended discussion 18:43:18 #endmeeting Generated by MeetBot 0.1.5 (http://wiki.ubuntu.com/meetingology)