Timr Testing Guy
Testing is like a good cake… Layered
Plans for Q3 2009
July 22nd, 2009 by timr
Here are some updates about our plans for this quarter (July-Sept 2009). We’ll be testing Firefox security releases along with the early releases for Firefox 3.6. Mobile is really heating up with early releases on Nokia devices, Windows Mobile devices, and SmartBooks.
The next area is making existing test tools and frameworks easier to use by the community. We have created a bunch of tools and leveraged existing tools. Now it is time to mkae them more user friendly and accessible to more members of the community. Some key areas here are more automated JS tests, QAC improvements, automated crash reproduction tools, and Litmus.next.
Next up is continuing our drive to automate manual tests. The manual Firefox smoke tests are now automated (Woo Hoo!!). It used to take 20 minutes platform to run these tests manually. The MozMill based automated smoke tests take only 2 minutes to run!!
The final focus for this quarter is developing community involvement. Here we are holding more meet-ups with potential and current community members. We have had great success with themes based meetups. Like crash reporting and analysis and testing web properties. We have also held meet-ups in Washington DC and Berlin Germany. And generally we want to do whatever we can to attract passionate new community members and continue to engage existing members.
Here is the detail list of things we are planning this quarter:
- Directly contribute to the successful releases of Firefox, Fennec, and prepare for 3.x testing
- Test Firefox 3.5.x, 3.0.x,
- Test Fennec milestones
- Test Major web sites (AMO, SUMO, mozilla.com, etc)
- Integrate WinMobile unit tests in build testing (tunit in particular)
- Create test tools and frameworks for capabilities of new features in 3.next: HTML5, HW Acceleration Gfx, Multi-process
- create manual BFTs for AMO, SUMO, and Spreadfirefox to ensure consistent regression testing.
- Make existing test tools and frameworks easier to use by community. Examples:
- Create automatic crash reproduction by stitching together existing tools from a manual process.
- Complete remaining work to fully automate the JS reftests – provides automated testing along with improved ability for JS devs to run and write tests
- Hook Fennec test results into new results server
- Fix top issues with QAC
- Port QAC for Fennec – make it easier for community to file bugs!
- Work with WebDev to fix most of the QMO P1 bugs (currently 14 P1s)
- Major improvements to Litmus – resolve performance issues, improve ability for web testers and l10n testers to use tool
- Make the coverage metrics more visible and reliable along with evangelize their use in the test development process
- Continue drive to automate manual tests
- Focus for this quarter will be the BFTs (Basic Functional Tests) and we will consider filling coverage gaps as we go along. We will target components based on the most critical features and lowest coverage. Specific components TBD.
- Integrate these tests into the buildbot testing framework. Joint goal with RelEng for review and landing!
- Automate key areas of the web testing BFTs with Selenium, for example. Key areas TBD.
- Develop Community involvement. Examples:
- Hold regular monthly meet-ups in Mountain View
- Attract new community members: Find 2-3 new community members to help with test development, test tools development, and/or multiple test execution sessions (ex: bug triage, fix verification, release testing)