== Meeting information == * #xubuntu-devel: Xubuntu testers session, 12 May at 19:08 — 20:29 UTC * Full logs at [[http://ubottu.com/meetingology/logs/xubuntu-devel/2017/xubuntu-devel.2017-05-12-19.08.log.html]] == Meeting summary == === Testing and verifying SRU bugfixes === The discussion about "Testing and verifying SRU bugfixes" started at 19:09. * '''SRU Questions''' (19:11) === Point release, Milestone and Daily testing === The discussion about "Point release, Milestone and Daily testing" started at 19:15. * '''Milestone and Daily Questions''' (19:18) === Package Testing === The discussion about "Package Testing" started at 19:23. * ''LINK:'' http://packages.qa.ubuntu.com/qatracker/testcases/1559/info is the testcase we provide for you to test Catfish. * ''LINK:'' https://bugs.launchpad.net/ubuntu-manual-tests/+filebug * '''Package Testing Questions''' (19:25) === Exploratory testing === The discussion about "Exploratory testing" started at 19:29. * '''PPAs''' (19:31) * ''LINK:'' https://launchpad.net/~shimmerproject/+archive/ubuntu/daily * ''LINK:'' https://launchpad.net/~xubuntu-dev/+archive/ubuntu/extras * ''LINK:'' https://launchpad.net/~xubuntu-dev/+archive/ubuntu/xubuntu-staging * ''LINK:'' https://launchpad.net/~xubuntu-dev/+archive/ubuntu/xfce4-gtk3 * '''Bug reporting''' (19:35) * '''Virtual Machines''' (19:36) * '''Exploratory Questions''' (19:36) * ''LINK:'' https://bugs.launchpad.net/launchpad/+bug/434733 === Git === The discussion about "Git" started at 19:59. * '''- Free Questions''' (20:00) == Vote results == == Done items == * (none) == People present (lines said) == * flocculant (172) * ali1234 (14) * akxwi-dave (6) * knome (4) * Spass (4) * ubottu (3) * meetingology (3) * Unit193 (2) * genii (2) == Full Log == 19:08 #startmeeting Xubuntu testers session 19:08 Meeting started Fri May 12 19:08:27 2017 UTC. The chair is flocculant. Information about MeetBot at http://wiki.ubuntu.com/meetingology. 19:08 19:08 Available commands: action commands idea info link nick 19:08 A couple of things to run through before we start properly. 19:08 The Xubuntu development tracker has a QA status tab, if there is anything that QA need to make testers aware of urgently, it will be on the notice board - https://dev.xubuntu.org/#tab-qa 19:09 Testing shouldn't be too onerous - do as little or as much as you can find the time for, all of it is appreciated by the whole Xubuntu team. 19:09 That said, being noticed by us can lead to invitations to Xubuntu QA and from there it is possible to get invited into Xubuntu Team where you get to be involved in the future of Xubuntu as a whole. For example, you get a vote in those mails you might have seen which start with [TEAM] 19:09 #topic Testing and verifying SRU bugfixes 19:09 First off we'll look at testing fixes that are targeted at released and still supported release(s) 19:09 For this you will need to be running whichever supported version the SRU is aimed at. We'll use an example - and let's use a fairly recent one - bug 1512120 19:09 bug 1512120 in thunar (Ubuntu Zesty) "[SRU] thunar crashes on file renaming" [High,Fix released] https://launchpad.net/bugs/1512120 19:10 You can see from that bug at comment 83 https://bugs.launchpad.net/ubuntu/+source/thunar/+bug/1512120/comments/83 19:10 that updated packages for both 16.04 and 16.10 were uploaded to the proposed repositories. To test the fix (and this applies to all SRUs) you need to enable that repository and then update the package list, either by using the Reload button or in a terminal with apt. 19:10 Once updated, you can reinstall the package (thunar) and test the new version against the testcase which should be present in the first comment of the SRU bug report. 19:10 Whether the patch fixes the bug for you, add a comment to the bug, mentioning which version you tested (dpkg -l 'package' ) and where (for example 16.04.2). Last but not least, change the verification tag to verification-done if the bug was fixed, or if the issue persists, use the verification-failed tag. 19:11 Once you have updated the package to check, you probably want to then disable the proposed repository or you will continue to receive updates to ANY package you have installed with a proposed update... 19:11 Not so hard - you've contributed to Xubuntu, and made life just a little bit better for other people. 19:11 #subtopic SRU Questions 19:15 #topic Point release, Milestone and Daily testing 19:16 First, we run testing for point releases (for example, 16.04.2) for an LTS release. We'll make calls for testing those as required. Note that these happen parallel to a normal development cycle and the tests need to be done with the current LTS release, not the release in development. 19:16 The second type is milestone testing: the Alpha, Beta and Release Candidate milestones of the current development release. We don't always take part in all milestones - usually more with LTS releases - but not necessarily so. 19:16 Before we go any further I want to make one very important point... 19:16 A milestone is JUST a snapshot of the current release in development. The milestones are useful for developers as it helps them and testers synchronize to one common state for a few days. 19:16 It is of no use once the milestone has passed. 19:16 The day after the milestone finishes, the archive is unfrozen and updates come through and the milestone is now out of date. Using old milestones isn't recommended or sensible - packages are updated very often during the development and it's better to just get the latest daily. (Milestones also aren't suitable for the general public as they aren't necessarily any stabler than any arbitrary daily. 19:17 ) 19:17 There are 2 main ways that you can test Xubuntu: either on a virtual machine or on hardware - the choice is up to you and what you have available. 19:17 We'd prefer people to be able to test on hardware because that let's us discover more "natural" bugs, but we also understand that you live in the real world and your PC doesn't belong to Xubuntu QA. 19:17 So, moving on then. We will call for testing of any milestone shortly before it's due, which gives people warning to put aside 20 minutes or so. There's no need for people to run through all the tests you might see for a milestone. Hopefully there will be more than just you running them. Obviously running tests which haven't been run is more important than doubling up. 19:17 You will see a link to the tracker in the mail, that will point you to somewhere like http://iso.qa.ubuntu.com/qatracker/milestones/374/builds/144361/testcases 19:17 On that page you'll see all of the tests needed to be run. Some of them are "Mandatory" and others are "Run once". If you see results listed against all the options - pick one. If there are gaps - pick those ;) 19:17 Run through the testcase you have picked EXACTLY how it is written - hopefully there will be no bugs and you can just mark it as Passed. 19:17 If you do find an issue, check the list at the bottom for known bugs - someone might have already seen it. When you find any bugs, add the bug number and report a Failed result. (Note that you can drag and drop bug links to the bug fields and the tracker will automatically parse the bug number correctly for you.) 19:18 Finally, the last type of testing is Daily testing. In daily testing you do exactly the same as with the milestone testing, you'll just find Daily testing on the tracker. The daily testing item runs all the way through the cycle refreshing overnight (2AM UTCish). 19:18 There is a simple run through of the ISO testing in general at https://wiki.xubuntu.org/qa/isotesting 19:18 #subtopic Milestone and Daily Questions 19:23 #topic Package Testing 19:23 The next section deals with how we can test packages. 19:23 In addition to the ISO tracker we looked at a moment ago, we also have a package tracker. This lists which packages we need testing and a testcase to run for each. Though it is likely this cycle that Thunar, at least, will see additional testcases added. 19:23 http://packages.qa.ubuntu.com/qatracker/testcases/1559/info is the testcase we provide for you to test Catfish. 19:24 As you can see it's pretty basic and lists exactly what we want. If you get to the end with no errors - pass it. If you find an issue - check at the end for already reported bugs, if it matches then you can use that bug number to Fail the test against. 19:24 If you find a new issue, then you need to report it. The simplest way is to use apport. To do this, open a terminal and run "ubuntu-bug packagename" - for example: ubuntu-bug catfish 19:24 You now need to wait for apport to finish and then you can complete the bug report in your browser when it is ready for you. When finished you can use the new bug number to fail your testcase against. It also helps to report bugs in Xfce packages upstream at https://bugzilla.xfce.org/ and link to the Xfce bug in Launchpad - we'll touch on that in the last section. 19:24 What happens when a bug gets fixed? 19:24 We'll mail testers and ask for that testcase to be re-run. Simply run through the testcase once more - hopefully this time you can pass it. 19:24 Sometimes you will find that a testcase is out of date - so what might appear to be a bug is in fact correct - we have a bug here - but it's with the testcase, and needs to be reported here: 19:24 https://bugs.launchpad.net/ubuntu-manual-tests/+filebug 19:24 Fill in a bug report for the testcase with as much information as is possible, especially give the testcase number, you can find this by using the 'Detailed information on the testcase' button - we will edit the testcase pretty quickly - we have tame Manual Testcase Admins amongst us. 19:25 #subtopic Package Testing Questions 19:29 #topic Exploratory testing 19:29 Exploratory testing is a term made up by Nick Skaggs (I assume) of Canonical which effectively means installing the development version and using it daily, for your daily tasks (and not a prespecified set of testcases). 19:29 Which is what I do. While I'm using it, I'm watching for bugs in our packages, I'm trying to check that visually we get no regressions and generally trying to ensure that we release the next supported version in as good a state as is possible. 19:29 I'll go through how I have my installs set up to give you some idea of what I've worked out to be the safest way to keep on rolling along. 19:30 I have 1 install of the last LTS which I keep around and 2 installs of the latest development version. 19:30 I have no data as such in any of these installs - all my data lives on other drives and is either mounted in fstab, linked to with a symbolic link or in the case of both Firefox and Thunderbird - the profile.ini points to the data. 19:31 I do have things like .ssh .bazaar which I have in my /home 19:31 The LTS is pretty much left alone until I need to check bugs or SRUs for that release. 19:31 The first of the development installs is what I use every day - I'm using it now. The other is my backup and has all the necessary links to enable me to boot it up and use it if necessary. I don't even boot it to keep it updated, I chroot to it and update packages there from this install. I tend to wait a day before updating if there's a package that could possibly cause problems and to make sure 19:31 I've been able to boot ... 19:31 ... into my daily install. 19:31 This second development install is without PPAs so I'm able to check what our default install is up to. 19:31 Personally I let grub run from my current development release - but I always have a USB handy in case grub goes pfffft... You can easily enough reinstall grub and let it be controlled from one of your other installs from the live desktop. 19:31 #subtopic PPAs 19:31 Xubuntu has a set of official PPAs which we use for testing new packages. These are: 19:32 https://launchpad.net/~shimmerproject/+archive/ubuntu/daily 19:32 https://launchpad.net/~xubuntu-dev/+archive/ubuntu/ppa 19:32 https://launchpad.net/~xubuntu-dev/+archive/ubuntu/extras 19:32 https://launchpad.net/~xubuntu-dev/+archive/ubuntu/xubuntu-staging 19:32 The last of these currently has no packages available for the Artful release, so it's listed now only as a possibility for use during this current cycle. 19:32 In addition we have one more PPA for the Xfce packages that have already been ported to GTK3: 19:32 https://launchpad.net/~xubuntu-dev/+archive/ubuntu/xfce4-gtk3 19:32 I have all 4 of these 'current' PPAs installed locally. These give me the most up to date Xfce packages available to Xubuntu. 19:33 So - that's how I'm set up locally. 19:33 But. 19:33 I am 1 person ... during a cycle we usually see about 5 or 6 people reporting on the trackers. We need more people to try to join this effort to make the Xubuntu we release is still what we want it to be. Getting bugs reported a week after we release helps no-one and with only a 9 month cycle for the most part it's unlikely that many of those will get fixed. More useful for us to see bugs reporte 19:33 d much much earlier. If in ... 19:33 ... the past you've found issues when you have installed a new version you'll understand that. 19:35 #subtopic Bug reporting 19:35 Reporting bugs using ubuntu-bug can be troublesome when you're using a package from a PPA, but for packages originating from the repo's you can follow the "ubuntu-bug packagename" route. 19:36 When you want to report a bug against a package originating in a PPA, you need to manually file it and also be prepared to supply information when asked for it by a developer. To do that, you can use: https://bugs.launchpad.net/ubuntu/+source/PACKAGENAME/+filebug which will error, so don't open it - a real example would be https://bugs.launchpad.net/ubuntu/+source/catfish/+filebug 19:36 Fill in the summary and then add as many details as you can. It is more likely that reporting this way developers will ask for more details. 19:36 To report an Xfce bug upstream, go to https://bugzilla.xfce.org/ and file your bug (check for duplicates first using the Search facility) the method is pretty similar to Launchpad - but is manual. 19:36 When you have your Xfce bug you can set your Launchpad bug to watch it, details for that can be found on the Ubuntu wiki at https://wiki.ubuntu.com/Bugs/Watches#Adding_a_Bug_Watch 19:36 #subtopic Virtual Machines 19:36 You could at a pinch run a virtual machine with the latest development version and use that for everything, but I suspect that the lure of not running inside a vm would be something to watch for - I know that I would soon end up running a supported release in that case. But the only person who can make that choice for you is you ;) 19:36 #subtopic Exploratory Questions 19:40 < Spass> small one - when I reported a bug with apport it default to Private, I assume it's secure to change it to Public? no sensitive data in the logs? 19:40 ok - this then is how I've dealt with that issue myself in the recent past 19:41 as I'm not coder enough to be sure what information is in a Private bug - and am therefore not sure whether to mark it as Public 19:42 i'd say in most cases it's ok to mark it public 19:42 I boot a virtual machine - replicate the bug - where possible - and then mark THAT one Public 19:42 then I've left the Private one be and let someone else decide 19:43 its the coredump that is sensitive mostly. that can have anything in it from previously used memory, and it is difficult to spot 19:43 knome: ack - I'm not going to make that call for other people - simply because I don't know what's there that's caused it to be marked Provate 19:43 flocculant, indeed :) 19:43 ali1234: thanks for that :) 19:43 private bugs are kind of annoying though 19:43 I thought it was likely coredump - but wasn't positive 19:43 if in doubt, developers/packagers/bug triagers should be able to help 19:44 if the bug has been retraced by apport and the stack trace makes sense you can delete the coredump and make it public after checking the other logs 19:44 also there's a bug with duplicate bugs and apport 19:44 let me find it, it's somewhat relevant 19:44 ali1234: right - what's likely to be the case is whoever reported isn't necessarily able to make the call 19:44 https://bugs.launchpad.net/launchpad/+bug/434733 19:44 Launchpad bug 764414 in apport (Ubuntu) "duplicate for #434733 private master bugs are confusing and lead to more duplicate filings" [Wishlist,Triaged] 19:45 apport does it automatically if it detects your stacktrace matches an existing one, even if the existing one is provate and you make yours public 19:46 yea - been there recentlyish - I think when talking with you about something :) 19:46 it is a pain for sure 19:46 probably, i know i've run into it before 19:46 :) 19:47 < Spass> Q: do you (devs) see the Private bug reports? 19:48 as far as I am aware only people in specific Launchpad teams can access Private bug reports 19:48 I know that I do not have the permissions 19:49 " The bug is reviwed by members of the ubuntu-bugcontrol team who will evaluate the bug and either keep the bug private, and assign specific developers, or remove the sensitive information if it is not helpful to solving the issue, and then mark it as public." 19:49 from https://askubuntu.com/questions/16135/why-did-apport-make-my-bug-report-private 19:50 There's some private ones I can see, but not all of them. I have no access to errors.ubuntu. 19:51 I have access to errors 19:51 members of the bug triage team can see all bugs 19:51 or at least the vast majority of them :) 19:51 Unit193: so is it likely you are a 'dev' on the package you can see when private? 19:51 ali1234: ack 19:52 flocculant: Do to being part of xubuntu-dev, I'm in bugcontrol. 19:52 aah ok 19:52 VERBOTEN 19:52 hi genii :) 19:53 "The Ubuntu Bug Control team (ubuntu-bugcontrol in Launchpad) is a subset of the Ubuntu Bug Squad" 19:53 * genii slides everyone fresh mugs of coffee and slips back into idling mode 19:53 [Q]: regarding tags - I assume that I must add "xubuntu-exp" to every bug report on LP and "ppa" when the package comes from PPA, right? 19:53 actually this page has a lot of info on it about coredumps etc: https://wiki.ubuntu.com/UbuntuBugControl 19:54 Spass: that was the plan - and still is - but seemingly not many people are running dev version during a cycle :) 19:54 it'd certainly not make the situation worse 19:55 we can report bugs against ppas now?? 19:55 ali1234: only by doing so manually - and then saying so 19:55 we don't do it much 19:55 hmm... pity 19:55 simpler to do it through bugzilla tbh 19:56 i got excited... reporting bugs against ppas has been a wishlist forever... 19:56 yea for sure it is a pity 19:57 ali1234: I'd be very very surprised if that was just you and I ;) 19:58 Spass: anything else? 19:59 flocculant: no, everything is clear now, thanks 19:59 #topic Git 19:59 Finally I just want to touch on a subject that's quite new to us at Xubuntu QA, building and testing Xfce packages built from the Xfce Git repositories. 20:00 During this cycle - as and when we're asked - we are likely to ask people to test fixes prior to them getting as far as PPAs. Duck out if you're not sure here. 20:00 Generally it's a painless experience as long as you've got the build dependencies installed - we used this method a couple of times towards the end of the Zesty cycle testing Thunar patches. 20:00 You'll need to install the build-essential package before the first time you build packages. After that, the process is simply the following: get the build-deps, grab the Git source, grab the patch and apply it, build it and then test it. 20:00 Whether we see more of this coming to us as testers via the mailing list is hard to say at the moment. 20:00 We'll not just now bother with questions on this last section, but if there are any more general questions - feel free to ask them now. 20:00 #subtopic - Free Questions 20:00 < knome> question: i'm confused by this two leaders thing... who should i contact if i want to start helping? 20:01 you can talk to either akxwi-dave or me 20:01 or in fact anyone in here is likely to be able to point you in the right direction 20:01 especially people who are members of !team 20:01 sigh 20:01 !team | will help 20:01 will help: akxwi-dave, bluesabre, dkessel, flocculant, jjfrv8, knome, krytarik, micahg, Noskcaj, ochosi, pleia2, slickymaster and Unit193 20:02 those people are all members of the Xubuntu Team and should be able to point you in the right direction 20:03 < knome> question: i'm worrying i don't have the required skills or equipment... please encourage me to help 20:04 If you are running Xubuntu - then you can help 20:04 I can - I drive a van 20:04 fresh eyes can help in many ways 20:04 I have no aptitude whatsoever for code - it's not needed to help us - though of course if you can - then that's an extra 20:06 akxwi-dave: for sure - it's more than likely that 2 people looking at the same screen will see slightly different things 20:06 and have different view points - some like knome and ochosi will notice the visual aspect more than I would 20:07 exactly flocculant :-) 20:08 q question: do you have the basics of QA or any other form of helping written down anywhere? 20:08 hang on 20:08 one more point on the last 20:09 while all of us might use - for example Parole - to play media 20:09 we'll have different workflows 20:09 and thus might find different bugs 20:09 I never use subtitles 20:09 so would be unlikely to see a bug there 20:10 (I'm looking at this from *my* point of view here) as I rarely run a simple testcase - but use them in the dev version daily 20:11 ok - so back to knome's question :) 20:11 akxwi-dave: can answer that one :p 20:12 first of all there is https://xubuntu.org/contribute/qa 20:13 andthis session will be put up on line shortly.. 20:14 that is very much specifically written by the Xubuntu team to cover what *we* want 20:14 the docs that is 20:14 also - there is the generic Ubuntu QA Team wiki page at https://wiki.ubuntu.com/QATeam 20:15 we also have a few pages on the Xubuntu wiki at https://wiki.xubuntu.org/start?do=index 20:15 there are also plans to do handy guides for testing, which are currently work in progress 20:16 to be honest 20:16 if there are things that people who do test - quietly in the background - think would be useful, then we are always receptive to suggestions 20:17 even more so when they do the work and ask for opinion 20:17 testing is something whihc is quite easy to get in to - and has great results 20:18 any more general testing questions out there? 20:19 [Q}: I'm not a member of "Xubuntu Testers" team on LP (https://launchpad.net/~xubuntu-testers). Should I be one and what's the reason for that? 20:19 well 20:20 while it's nearly always the case that testing calls go out to the development mailing list 20:20 I have in the past just contacted people in the Testers LP team directly 20:21 also it's a social thing - we have at least some idea that we have 20 or 30 or 40 people testing for us 20:21 and people who show up on LP or on the trackers - get noticed by us 20:22 +1 member on Xubuntu Testers then :) 20:22 then get asked to join the Xubuntu QA Team itself - and from there it's possible to become part of Xubuntu Team and be part of the group of people marking out Xubuntu's direction 20:23 < Unit193> Sooo, is there a point to me jumping early? I don't use parole, lightlocker, whiskermenu, gnome-software,thunderbird, snaps, sgt-puzzles, etc... 20:24 by jumping early - I'll assume you mean upgrading zesty to artful (or a to b, b to c) 20:24 yes there is 20:25 while you might not specifically use all of the packages, generally you're using Xubuntu 20:25 so issues with the main 'whole' could come to light 20:25 and also 20:26 it could happen that you removing something from our seeded defaults causes you issue where it might not the rest of us 20:27 we don't know how everything installable works in the whole for us - we can only test so much 20:27 I for instance am more likely to use any music player that's around - I find things on the rare times I do use our defaults 20:28 ok - so 90 minutes later - any other questions? 20:28 going 20:28 going 20:29 #endmeeting Generated by MeetBot 0.1.5 (http://wiki.ubuntu.com/meetingology)