聂长海;

• 1:南京大学软件新技术国家重点实验室暨计算机科学与技术系

摘要(Abstract)：

经过30多年的发展,组合测试已经发展成为1种独立的和颇具影响的软件测试方法。该方法利用组合覆盖表作为测试用例集,可以有效检测软件系统中各种因素及其相互作用而促发的各种故障。论文在充分分析已有文献的基础上,结合作者在该领域的研究实践,首次系统介绍组合测试的演化发展,总结出其中的关键问题、方法、重要应用和未来发展方向,为组合测试的进一步研究和发展提供支持和参考。

关键词(KeyWords)： 软件测试;组合测试;覆盖表;测试用例生成;故障诊断

Abstract:

Keywords:

基金项目(Foundation): 高等学校博士学科点专项科研基金资助项目(20130091110032)

作者(Author): 聂长海;

Email:

DOI:

参考文献(References)：

• [1]NIE C,WU H.Combinatorial testing repository[EBOL].(2016-05-20)http://gist.nju.edu.cn/ct_repository/.
• [2]COHEN D M,DALAL S R,FREDMAN M L,et al.The AETG system:an approach to testing based on combinatorial design[J].IEEE Transactions on Software Engineering,1997,23(7):437-444.
• [3]NIE C,LEUNG H.A survey of combinatorial testing.ACM computing surveys[J].2011,43(2):11:1-11:29.
• [4]DALAL S R.,MALLOWS C L.Factor-covering designs for testing software[J].Technometrics,1998,40(3):234-243.
• [5]DALAL S R,JAIN A,KARUNANITH N,et al.Model-based testing in practice[C]//Proceedings of the International Conference on Software Engineering.Los Angeles,USA,IEEE,1999:285-294.
• [6]DALAL S R,JAIN A,KARUNANITHI N,et al.Model-based testing of a highly programmable system[C]//Proceedings of 9th International Symposium on Software Reliability Engineering.Paderborn,Germany:[s.n.],1998:174-178.
• [7]LOTT C,JAIN A,DALA S L.Modeling requirements for combinatorial software testing[C]//Proceedings of the first international workshop on Advances in Modelbased,ACM SIGSOFT Software Engineering Notes.Missouri,USA:IEEE,2005,30(4):1-7.
• [8]WILLIAMS A W.Software component interaction testing:coverage measurement and generation of configurations[D].Canada:University of Ottawa,2002.
• [9]KRISHNAN R,KRISHNA S M,NANDHAN P S.Combinatorial testing:learnings from our experience[J].ACM SIGSOFT Software Engineering Notes,2007,32(3):1-8.
• [10]GRINDAL M,OFFUTT J.Input parameter modeling for combination strategies[C]//Proceedings of IASTED International Conference on Software Engineering.Innsbruck.Austria:IEEE,2007:255-260.
• [11]CZERWONKA J.Pairwise Testing in real world:practical extensions to test case generator[C]//Proceedings of 24th Pacific Northwest Software Quality Conference.Portland,Oregon,USA:IEEE,2006:419-430.
• [12]NIE C,CHEN S,LEUNG H,et al.A case study of adaptive combinatorial testing[C]//Proceedings of IEEE 37th Annual Computer Software and Applications Conference Workshops.Kyoto,Japan:IEEE,2013:47-52.
• [13]NIE C,CHEN S,LEUNG H,et al.Adaptive combinatorial testing[C]//Proceedings of 13th International Conference on Quality Software.Nanjing,China:[s.n.],2013:284-287.
• [14]GRINDAL M,LINDSTROM B,OFFUTT J,et al.An evaluation of combination strategies for test case selection[J].Empirical Software Engineering,2006,11:583-611.
• [15]FOUCHE S,COHEN M.B,PORTER A.Towards incremental adaptive covering arrays[C]//Proceedings of The Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering.Dubrovnik,Croatia:ACM,2007:557-560.
• [16]NIE C,WU H,LIANG Y,et al.Search based combinatorial testing[C]//Proceedings of The 19th Asia-Pacific Software Engineering Conference.Hong Kong:IEEE,2012:778-783.
• [17]WU H,NIE C,KUO F,et al.A discrete particle swarm optimization for covering array generation[J].IEEE Transactions on Evolutionary Computation,2015,19(4):575-591.
• [18]WU H,NIE C.An overview of search based combinatorial testing[C]//Phil McMinn eds.The 7th International Workshop on Search Based Software Testing(SBST).Hyderabad,India:ACM,2014:27-30.
• [19]QU X,COHEN M B.,WOOLF K M.Combinatorial interaction regression testing:a study of test case generation and prioritization[C]//Proceedings of IEEE International Conference on Software Maintenance.Paris,France:IEEE,2007:255-264.
• [20]QU X,COHEN M B,ROTHERMEL G.Configuration-aware regression testing:an empirical study of sampling and prioritization[C]//Proceedings of ISSTA.Seattle,WA,USA:ACM,2008:75-86.
• [21]BRYCE R C,COLBOURN C J..Prioritized interaction testing for pair-wise coverage with seeding and constraints[J].Information and Software Technology,2006,48(10):960-970.
• [22]BRYCE R C,COLBOURN C J.Test prioritization for pairwise Interaction coverage[J].ACM SIGSOFT Software Engineering Notes,2005,30(4):1-7.
• [23]BRYCE R C,MEMOM A M.Test suite prioritization by interaction coverage[C]//Proceedings of DomainSpecific Approaches to Software Test Automation Workshop.Dubrovnik,Croatia:[s.n.],2007:1-7.
• [24]WU H,NIE C,KUO F.The optimal testing order in the presence of switching cost[J].Information and Software Technology,2016,80:57-72.
• [25]YILMAZ C,COHEN M B,PORTER A A.Covering arrays for efficient fault characterization in complex configuration spaces[J].IEEE Transaction on Software Engineering,2006,32(1):20-34.
• [26]COLBOURN CJ,MCCLARY D W.Locating and detecting arrays for interaction faults[J].Journal of Combinatorial Optimization,2008(15):17-48.
• [27]MARTINEZ C,MOURA L,PANARIO D,et al.Algorithms to locate errors using covering arrays[J].LATIN,2008:504-519.
• [28]NIE C,LEUNG H.The minimal failure-causing schema of combinatorial testing[J].ACM Transactions on Software Engineering and Methodology,2011,20(4):15:1-15:38.
• [29]BACH J,SCHREODER P J.Pairwise testing:a best practice that isn’t[C]//Proceedings of 22nd Pacific Northwest Software Quality Conference.USA:IEEE,2004:180-196.
• [30]SALEM A M.A software testing model:using design of experiments(DOE)and logistic regression[D].Florida:Florida Institute of Technology,2001.
• [31]YILMAZ C,FOUCHE S,COHEN M B,et al.Moving forward with combinatorial interaction testing[J].IEEE Computer,2014,47(2):37-45.
• [32]KHALSA S K,LABICHE Y.An orchestrated survey of available algorithms and tools for combinatorial testing[C]//Proceedings of International Symposium on Software Reliability Engineering.Naples,Italy:IEEE,2014:323-334.
• [33]ANAND S,BURKE E K,CHEN T Y,et al.An orchestrated survey of methodologies for automated software test case generation[J].Journal of Systems and Software,2013,86(8):1978-2001.
• [34]GRINDAL M,OFFUTT J,ANDLER S F.Combination testing strategies:a survey[J].Software Testing,Verification and Reliability,2005,15(3):167-199.
• [35]COLBOURN C J,DINITZ J H,STINSON D R.Applications of combinatorial designs to communications,cryptography and networking[J].Surveys in Combinatorics,1999,22:37-100.
• [36]COLBOURN C J,VANOORSHOT P C.Applications of combinatorial designs in computer science[J].ACM Computing Surveys,1989,21(2):223-250.
• [37]严俊,张健.组合测试:原理与方法[J].软件学报,2009,20(6):1393-1405.YAN Jun,ZHANG Jian.Combinatorial testing:princeple &methods[J].Chinese journal of software,2009,20(6):1393-1405.(in Chinese)
• [38]聂长海,组合测试[M].北京:科学出版社,2015:4.NIE Changhai.Combinatorial testing[M].Beijing:Science press,2015:4.(in Chinese)
• [39]聂长海,蒋静.覆盖表生成的可配置贪心算法优化[J].软件学报,2013,24(7):1469-1483.NIE Changhai,JIANG Jing.Configurable greedy algorithm for covering array generation[J].Chinese journal of software,2013,24(7):1469-1483.(in Chinese)
• [40]LI X,DONG Z,WU H,et al.Refining a randomized post-optimization method for covering arrays[C]//Proceedings of ICSTW 2014.Cleveland,OH,USA:IEEE,2014:143-152.
• [41]NIC C,WU H,NIU X,KUO F,et al.Combinatorial testing,random testing,and adaptive random testing for detecting interaction triggered failures[J].Information and Software Technology,2015,62:198-213.
• [42]钮鑫涛,聂长海,CHAN Alvin.组合测试故障定位的关系树模型[J].计算机学报,2014,37(12):2505-2518.NIU Xintao,NIE Changhai,CHAN Alvin.Relation tree model of fault location for combinatorial testing[J].Chinese Journal of computer,2014,37(12):2505-2518.(in Chinese)