The World Quality Report is the result of ongoing collaboration between Capgemini, Sogeti and HP Software. The report is based on a global survey of more than 1,200 CEOs, CFOs, CIOs, IT directors/managers and quality assurance directors/managers from around the world.
Rahman et al. have used a fixcache/bugcache approach to predicting which files should be singled out for code inspection. I have written about using a similair approach for regression test selection (here and here). Their main conclusion is:
When used for inspection, FixCache gets most of its
power by predicting that les that recently experienced
a bug x, do in fact contain additional, latent bugs.
So basically, what they have found is that bugs tend to occur grouped in the same file. I guess there is some profound truth here. But is this just some kind of random statistical property of software artefacts or does it tell us something fundamental about software development?
@inproceedings{citeulike:9675829,
author = {Rahman, Foyzur and Posnett, Daryl and Hindle, Abram and Barr, Earl and Devanbu, Premkumar},
citeulike-article-id = {9675829},
citeulike-linkout-0 = {http://macbeth.cs.ucdavis.edu/fse2011.pdf},
keywords = {20110817},
posted-date = {2011-08-17 10:21:34},
priority = {0},
title = {{BugCache} for Inspections : Hit or Miss?},
url = {http://macbeth.cs.ucdavis.edu/fse2011.pdf},
year = {2011}
}
@article{citeulike:7426424, abstract = {Background: The fix-cache approach to regression test selection was proposed to identify the most fault-prone files and corresponding test cases through analysis of fixed defect reports. Aim: The study aims at evaluating the efficiency of this approach, compared to the previous regression test selection strategy in a major corporation, developing embedded systems. Method: We launched a post-hoc case study applying the fix-cache selection method during six iterations of development of a multi-million {LOC} product. The test case execution was monitored through the test management and defect reporting systems of the company. Results: From the observations, we conclude that the fix-cache method is more efficient in four iterations. The difference is statistically significant at alpha = 0.05. Conclusions: The new method is significantly more efficient in our case study. The study will be replicated in an environment with better control of the test execution.},
address = {Los Alamitos, CA, USA},
author = {Engstrom, Emelie and Runeson, Per and Wikstrand, Greger},
citeulike-article-id = {7426424},
citeulike-linkout-0 = {http://doi.ieeecomputersociety.org/10.1109/ICST.2010.40},
citeulike-linkout-1 = {http://dx.doi.org/10.1109/icst.2010.40},
doi = {10.1109/icst.2010.40},
isbn = {978-0-7695-3990-4},
journal = {Software Testing, Verification, and Validation, 2008 International Conference on},
keywords = {20110817, 20111003-irop},
pages = {75--78},
posted-date = {2010-07-08 01:34:23},
priority = {0},
publisher = {IEEE Computer Society},
title = {An Empirical Evaluation of Regression Testing Based on {Fix-Cache} Recommendations},
url = {http://dx.doi.org/10.1109/icst.2010.40},
volume = {0},
year = {2010}
}
@inproceedings{citeulike:7426413, abstract = {This paper presents a simple method that computes test case coverage information from information on what files were updated to fix a fault found by the test case. It uses a cache to monitor fault-prone files and recommends test cases to rerun to cover updated files. We present an evaluation of the method during two months of development of a large, industrial, embedded, real-time software system. Our results show that the method is effective, reaching weekly cache hit rates in the range 50-80\%.},
address = {Los Alamitos, CA, USA},
author = {Wikstrand, Greger and Feldt, Robert and Gorantla, Jeevan K. and Zhe, Wang and White, Conor},
booktitle = {ICST '09: Proceedings of the 2009 International Conference on Software Testing Verification and Validation},
citeulike-article-id = {7426413},
citeulike-linkout-0 = {http://portal.acm.org/citation.cfm?id=1548215},
citeulike-linkout-1 = {http://doi.ieeecomputersociety.org/10.1109/ICST.2009.42},
citeulike-linkout-2 = {http://dx.doi.org/10.1109/icst.2009.42},
citeulike-linkout-3 = {http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4815362},
doi = {10.1109/icst.2009.42},
isbn = {978-0-7695-3601-9},
journal = {Software Testing, Verification, and Validation, 2008 International Conference on},
keywords = {20110817},
location = {Denver, CO, USA},
month = apr, pages = {299--302},
posted-date = {2010-07-08 01:20:26},
priority = {0},
publisher = {IEEE Computer Society},
title = {Dynamic Regression Test Selection Based on a File {Cache—An} Industrial Evaluation},
url = {http://dx.doi.org/10.1109/icst.2009.42},
volume = {0},
year = {2009}
}
When you review documents, test applications etc it is very easy to get stuck on shallow problems and give comments which are not really helpful for the developers. Using checklists can be one way to force yourself to look beyond the most obvious things. I recently found two good checklists/protocols.
A Waterfall is Fine – for Watching
Wordle for March 2012
Agile Offshoring in Practice
Waterfall got it right… almost