In its 22nd year, UKSMA, the United Kingdom Software Metrics Association
hosts their conference with
COSMIC, the Common Software Measurement International Consortium
with the support of
Unicom Seminars Ltd
International Conference on Software Metrics and Estimating
to be held in London, 27th/28th October 2011

Biographies and Abstracts

Keynote: Martin Shepperd, Brunel University, UK Prof. Shepperd has the chair of Software Technology at Brunel University. He received a PhD in computer science from the Open University in 1991 for his work in measurement theory and its application to empirical software engineering. He has published more than 150 refereed papers and three books in the areas of software engineering and machine learning. Previously he worked for a number of years as a software developer for a major UK bank.

Keynote: Carolyn Mair, Southampton Solent University, UK Dr Mair is a senior lecturer in psychology at Southampton Solent University. She is a Chartered Psychologist and Chartered Scientist. Carolyn obtained her PhD in Cognitive Neuroscience from Bournemouth University for her investigations into spatio-temporal aspects of visual memory and completed her post-doc at Brunel University. Carolyn has previously worked as a graphic designer, portrait artist, dressmaker and teacher of English as a foreign language.

Abstract: ‘Meta-cognition, Confidence and Bias in Software Estimating’ Effective prediction, for example of project costs, is an essential aspect of software development. Although considerable research has been devoted to this topic, the role of the human experts who make and are responsible for predictions, has been under-emphasised. We focus on how metacognitive (thinking about thinking) awareness impacts on prediction and confidence (uncertainty assessment) as confidence plays a crucial role in prediction. For example a manager may be 90% confident that the system testing will be completed within 4 weeks. Over-confidence may be as great a threat as over-optimism and the two are inter-linked. Our aim is to improve the prediction practices of software professionals by reducing over-confidence and over-optimism (bias) which are recurring problems. The talk will cover: • the meaning of a prediction, that it is a probabilistic statement which implies two components: (i) the predicted value and (ii) the degree of confidence in the prediction • evaluating the quality of a prediction includes: (i) error (ii) bias and (iii) variance or scatter • cognitive sources of over-confidence and over-optimism e.g. the planning fallacy, anchoring effects and the peak-end rule • de-biasing strategies through enhancing meta-cognitive awareness.

Keynote 2: Charles Symons President The Common Software Measurement International Consortium (COSMIC) UK cr.symons@btinternet.com  Charles Symons is semi-retired after 50 years in computing. A graduate in physics, he has worked as a scientific programmer, managed large data centres, been responsible for IS standards-setting, and has led consulting studies on IS Strategy and improving the performance of the IS function in many parts of the world. He is currently President of the Common Software Measurement International Consortium (COSMIC), with a specialist interest for the last 25 years in improving the use of measurement and estimating for software activities.

Abstract: ‘Accounting for Non-Functional Requirements in Productivity Measurement, Benchmarking & Estimating’ Measurements of software project performance and the use of such measurements for benchmark comparisons and for estimating all require a measure of the size of software that should ideally reflect all the requirements, functional and non-functional. Counts of SLOC arguably do reflect all requirements but suffer from other disadvantages as a reliable and useful size measure. ‘Function Point’ measures have used a VAF or TCA to account for the influence of non-functional requirements (NFR) and IFPUG is currently working on a SNAP measure with the same goal. But there is no clear definition of what is an NFR, there are very many different types of NFR, and they vary enormously in their relative importance. So attempts to construct a universally-usable size index for NFR will not prove satisfactory. A re-think is needed. First, many NFR such as for maintainability, usability, security, etc actually evolve into software functions as the design progresses, so their functional size can be measured. Second, I propose an unambiguous definition of ‘functionality’ which leads to a very clear understanding of NFR (all other requirements). The result is that all requirements can be classified as ‘functional’ or non-functional’, the latter in five categories: • Quasi NFR that evolve into software functions, e.g. ‘the software must be easy to use...’ • Software constraints, that do not evolve into software functions, e.g. ‘the software must be written in C♯’ • Technology constraints, e.g. ‘the system must execute on PC and Apple platforms’ • Project constraints, e.g. ‘the project must be completed by the end of the year and cost less than $1M’ • Other (System) Deliverables, e.g. training, documentation We now have a precise way of distinguishing requirements that contribute to software functional size (Functional requirements and QNFR) from other requirements that affect project effort but that do not contribute to functional size and/or that may not require effort proportional to functional size. Another problem is that size measurements and other project data collected to understand project performance should be usable directly for benchmarking and estimating. I will refer to data collected by some publicly-available methods for performance measurement, benchmarking and estimating to show the gaps and inconsistencies in the classes of NFR that need to be corrected and filled to meet all three needs. I will propose a draft structure of NFR that meets all three needs and that should be a basis for standardization.

Luca Santillo Agile Metrics /ETS, Italy Via Pollastrini, 7 – 00062 Bracciano (RM) luca.santillo@gmail.com Luca Santillo (MSc) is a recognized software measurement expert. He publishes guidelines and innovative approaches of Functional Size Measurement in fields such as those of Data Warehousing, Web-based systems, Service Oriented Architectures, Agile development, and Early & Quick estimates. His M.Sc. in Physics allows him to apply exact methods to the emerging field of software measurement within software engineering, with innovative insights and measurement divulgation among practitioners. He chairs several working groups with national and international bodies, related to Function Points and Benchmarking. To-date, he’s President of the Italian Software Metrics Association (GUFPI-ISMA), Italian member of the International Advisory Council of the Common Software Measurement International Consortium (COSMIC), and one of the Directors, and Honorary Treasurer, of the International Software Benchmarking Standards Group (ISBSG). Luca Santillo is a Function Point certified specialist in both COSMIC and IFPUG methods. As a consultant and specialized trainer, he helped dozens of medium-to-large companies to apply FPA, productivity factors assessment, and quantitative project management, to software development, in Italy and abroad. He published over 50 contributions to international conferences and books, including ESCOM, FESMA, IWSM, MENSURA, and COSMIC and IFPUG books, including awarded papers on Uncertainty Propagation in Software Estimation and Functional Size Measurement in Agile projects

Abstract: ‘Towards a Sound Adoption of Measurement and Metrology in Software Engineering’ Measurement in software engineering has been developing since mid 70’s of last century, as a variegated collection of trials and proposals from several practitioners and researchers, until the recent standardization of FSM (Functional Size Measurement) with the ISO/IEC 14143 standards and technical reports series. (Independently, “metrics” had been published on software quality in 1991 in ISO/IEC 9126, revised 2001.) When compared to measurement and metrology basics in hard science, any software measurement method shows significant gaps. The current situation in fact resembles the historical period that separated the locally-defined myriad of length measurement units in the ancient times and the globally-defined unit of the meter as the length base unit in the International System of Units (SI) in the modern era. Technology could not have progressed so far as in our times without a clear set of definitions of measurable quantities, a framework of measurement units and adequate procedures to measure quantities in real-world phenomena accordingly to such framework. The design of measurement concepts and metrology notions in science has not been straightforward – it did take centuries to achieve a worldwide accepted definition for units like the meter for length, the second for time, and so on. Moreover, such definition process is continuously improving, although nowadays standard definitions for basic quantities in physics have reached significantly high levels of precision, with negligible errors from the perspective of most applications. If we look for a similar precision in software measurement, and in its industrial adoption, metrology principles and measurement units must be addressed. This work introduces basic metrology concepts, good-versus-bad examples of measures in software engineering, and a possible extension of the SI to include software measurement. An overview of past attempts to develop software measurement standards (aka etalons), the topics of uncertainty and of dimensional and dimensionless units, a metrological comparison of current FSM methods, and the analysis of what actions may or should be undertaken to achieve.

Gökçen Yılmaz Informatics Institute METU 06531, Ankara Turkey gokcen@ii.metu.edu.tr  Gökçen Yılmaz is an MSc student and research assistant in Middle East Technical University, Informatics Institute, Department of Information Systems. She has BSc degree in Systems Engineering from Yeditepe University. Her research interests include: software size measurement and estimation, software measurement verification, software project management.

Abstract: ‘The Effect of the Quality of Software Requirements Document on the Functional Size Measurement’ Functional size measurement methods have been criticized based on discrepancies among different measurers and accuracy of measurements. It is assumed that measurement variation among measurers will decrease the reliability of the measurements. This assumption discourages software organizations from adopting functional size measurement. According to our previous studies; sources of measurement variations are not only quality of measurers and the measurement approaches but also the quality of software artifacts. In this study we aim to observe how errors in requirement specification affect the accuracy of COSMIC size measurement. We aim to compare the measurement errors introduced in the analysis phase and those occur in the measurement phase. At the end of our case study a check list will be created for defect detection based on common errors found in the case study. We conducted a two-step case study. In the first step, a software requirements specification (SRS) document and a simple data analysis was prepared for a given requirements statement. Two SRSs were generated; first one was prepared by the participants and the second was prepared by the measurement experts. They are referred as Generated SRS (G-SRS) and Reference SRS (R-SRS) respectively in this paper. Same measurement experts measured these two SRSs and created measurement keys for both G-SRS and R-SRS and they are called G-Key and R-Key respectively. In the second step a different group of participants measured the G-SRS using COSMIC measurement method. We evaluated each measurement report against both G-Key and R-Key. We categorized the most commonly made errors based on the defect causes as defined in the COSMIC Guideline for Assuring the Accuracy of Measurements. We mapped the errors to these causes as:     • Quality of the measurer(s)     • Quality of the software artifacts     • Quality of the measurement process We calculated the COSMIC size of errors for each factor and their percentage in the total size of errors. According to our findings; if the effect of quality of software artifacts is ignored , reliability of size measurements seems to be mostly dependent on quality of measurers. However, when we consider the analysis and measurement phases as a whole we observed that quality of software artifacts (SRS in this study) is highly critical for the accuracy of size measurements. To improve the reliability of FSM size measurements, software artifacts should be prepared by software engineers who possess a measurement point of view. In order to detect errors in size measurements, measurements results should be reviewed. Compiling the most commonly made errors, we generated a check list which can be used in junction with the lists in COSMIC Guideline for Assuring the Accuracy of Measurements. As future work we will automate the use of this check list developing a tool to facilitate the reviews on software artifacts for size measurement purposes.

Alan Cameron Director David Consulting Group Ltd UK Alan Cameron, Director of David Consulting Group Ltd, has 25 years of industry experience both as customer and outsource supplier. He has managed requirements in major government projects, devised and implemented measurement programmes for multi-million pound contracts, and as part of the IT strategy group has helped clients to re-focus their business and move to more effective use of their software. Prior to joining David Consulting Group, Alan led the EMEA Sizing, Estimating and Process Group for HP Application Services.

Abstract: ‘Here and now or the Value Tetrahedron?’ This paper introduces the concept of the Value Tetrahedron, one which has been developed for business use of software metrics. The Value Tetrahedron enables a business to understand the balance between technical debt and software development performance based on software metrics. Technical debt is the inherent indebtedness of an organization due to deviations from technical and architectural standards and uncleared known and unknown defects. The concept of the Value Tetrahedron allows an organization to make informed decisions about the level of technical debt that it is willing to carry within its systems and portfolio. In an industry driven increasingly by cost and schedule pressures, it is no longer sufficient to consider software development projects only in terms of the time, cost and quality considerations commonly identified at project inception. The extra Technical Debt dimension must be quantified at the planning stage and tracked thereafter. At the same time support teams must quantify and track total technical debt for each application. Using the ideas inherent in the Value Tetrahedron an organisation can determine the acceptable cost of deviations from quality, architectural and other standards as a result of one or many releases of software. Once a pre-determined limit is reached then action can be taken to re-balance the debt. The paper describes the concepts behind the tetrahedron and how these can be measured in terms of software metrics. It will then look at the stages in debt control and discuss ways in which organisations can address this pernicious problem. The concept of stages of control will be discussed, covering Organisational Management, Application Support and Software Development. This will cover planning, estimating and debt management

Shekoufeh Kolahdouz-Rahimi Department. of Informatics King’s College London UK Shekoufeh.kolahdouzrahimi@kcl.ac.uk; kevin.lano@kcl.ac.uk; iman.poernomo@kcl.ac.uk Shekoufeh Kolahdouz Rahimi is a doctoral candidate in Computer Science in the Software Modelling and Applied Logic (SMAL) Group at Kings College London. She received her MSc in Computing and Internet Systems from the King’s College London, UK and her BSc in Hardware Engineering in her home town Esfahan, Iran. Her research interest include Specification, Verification and Evaluation of Model Transformation

Abstract: ‘Goal-Oriented Measurement: Comprehensibility of Model Transformations’ with Kevin Lano, and Iman Poernomo, Department. of Informatics, King’s College London, Strand, London, UK Model transformations are employed for variety of reasons within model driven development: to improve model quality, to systematically apply design patterns for refactoring, to map models from one language to another. A large number of model transformation languages and tools have been developed across the research community. However, there are no guidelines on how to select appropriate notations for particular model transformation tasks, and no comprehensive comparisons of the relative merits of particular approaches In order to provide such comparisons, it is important to develop metrics that work across the wide range of paradigms used to define transformations. This trans-paradigmatic nature of comparison renders it a hard problem. This paper suggests a possible approach to the problem, focusing on the general property of comprehensibility, showing how the Goal/Question/Metric (G/Q/M) work can be used to specialise an approach. Several feature of model transformation are discussed in previous works, however the Comprehensibility feature of model transformation is not considered. Comprehensibility, refers to the ability in which specification and implementation of model transformation can be understood and analysed. A comprehensive specification is crucial for an efficient maintenance process.

Dr Simon Wright Director at SymTech Ltd UK email: simon@symtech.ltd.uk tel: 07711 846554 Simon is an independent consultant whose experience has been gained in major blue-chip companies and public sector bodies throughout the world. Recent assignments include six months working for the Russian Atomic Energy Authority, eighteen months working with Saab Technologies and three months on the Trusted Boarders project. He graduated with a BSc and PhD in Chemistry from the University of London. After eight years as a research chemist he moved into software engineering, specifically mathematical modelling and simulation at GEC-Marconi. Within GEC-Marconi Simon held several managerial positions and built a successful business based upon an in house modelling tool called COSMOS. When this business was “restructured” as part of a company wide restructuring Simon moved into the commercial software tool arena and joined Integrated Chipware as their European Director of Software Engineering. Assignments at Nokia, Airbus and BAE Systems enabled him to develop an industry wide reputation as a high achiever. Now as an independent consultant the key to his success has been his ability to gain the trust of his clients through technical competence coupled with his excellent client facing, customer-focused communication skills.

Abstract: ‘Some Pitfalls of Three Point Estimation and How to Avoid Them’ The three-point estimation technique is widely used in management and information systems to estimate parameters such as size, effort and/or duration. The three points are usually converted into a probability distribution and then combined with other three point estimates to give an overall estimate of size, effort or duration. Care has to be exercised in transforming the three points into a probability distribution because without understanding the consequences of, say, adopting a triangular distribution it is easy to fall into a pitfall that could guarantee failure. For example a three point size estimate may take the form:    A, the best-case estimate = 30 points    C, the most likely estimate = 45 points    B, the worst-case estimate = 90 points Since this estimate is asymmetric, (positively skewed), it is not appropriate to model it as a Normal distribution and without any a priori information most people would model this size distribution as a triangular distribution. /> The pitfall arises due to the fact that the “most likely” estimate is neither the average, (the mean size) nor the “most probable”, (the median size), and in this case it could result in a 75% chance of an overrun due to an underestimate of the actual size. Care must also be exercised in combining three point estimates because the bias can be compounded. For example a three point effort estimate may take the form:    A, the best-case estimate = 30 staff/hrs    C, the most likely estimate = 45 staff/hrs    B, the worst-case estimate = 90 staff/hrs If a task was composed of three of these effort estimates and again a triangular distribution was assumed then the deterministic effort duration would be three times the most likely, 3*45 = 135 staff/hrs. In fact the probability of completing the task in 135 staff/hrs is 10%, in other words there is a 90% chance of an overrun; an almost guarantee of failure. OOnce known and understood these pitfalls can be mitigated. For example a double triangle distribution might be chosen as a better approximation of the probability distribution. In addition Monte Carlo simulation of the tasks can be used to model the project and determine the most likely size, effort or duration. The use of the double triangle distribution and Monte Carlo simulation to avoid there pitfalls is discussed and recommendations for their use are given.

Roberto Meli CEO, DPO Srl ITALY Robert Meli graduated in Computer Science in 1984. In 1996 he became CEO of DPO Srl, a qualified supplier of services in the ICT market. DPO’s excellence areas are: Software Measurement and Estimation, Requirements Management, Project & Risk Management. DPO provides specialised services and products to promote the continuous evolution of production and management processes for both public and private organizations. During the past 20 years Meli has developed focused competences in Project Management and Software Measurement areas and has written papers for technical magazines and international conferences. Certified Function Points Specialist (CFPS) from 1996 to 2004, he is a consultant and lecturer in training courses on Project Management and Software Measurement for many major Italian companies and public organizations. He developed the Early & Quick Function Point Analysis method, managing the implementation of the Sfera product. In the last 15 years he had the following roles: chairperson of the Board of Directors of GUFPI-ISMA (Gruppo Utenti Function Points Italia - Italian Software Measurement Association), coordinator of the GUFPI - ISMA Counting Practices Committee; Italian delegate to the MAIN (Metrics Association’s International Network), Chairperson of the COSMIC Measurement Practices Committee and Conference Chairperson of SMEF event (Software Measurement European Forum), one of the leading European events in this area. He has managed the GUFPI-ISMA working group which released the Italian Guide Lines for the Contractual Usage of Function Points. This document has been then used to feed the Italian Government ICT Authority Guide Lines to be used by all Italian Public Administrations. Currently, he is President of the Simple Function Point Association (SiFPA).

Abstract: ‘Simple Function Points’ The Simple Function Point method has been developed by DPO in 2010, used since the late part of the year and presented at SMEF2011. Simple Function Point is not an estimation method (like Early & Quick FP) but a real FSMM compliant with ISO 14143 principles. It is totally compliant with the IFPUG method at the results level. In other words, if applied to the same software application portfolio they both give the same numbers. In case of application of both methods to more than 700 ISBSG counts valued more than 290’000 UFP, the error was less than 0,4%. All the resources and contractual frameworks developed for IFPUG are valid for Simple FP as well, starting from the ISBSG productivity data base to unitary rates of contracts. The usage of the new method reduces cost, time and disputes, the translation of an entire measured application portfolio is immediate. Training course is half than the usual IFPUG one but the time for becoming proficient in usage of the measurement rules is maybe one tenth. Measurement may be done by production people in alternative to measurement specialists. The original idea was to test if effort in developing or maintaining software was better correlated to the number and internal complexity of BFC or if it was enough to consider only the “number” of BFC in the cost model. DPO’s research (made on publicly available data) has demonstrated that the first hypothesis was not true. The precision of the statistical correlation is not higher in the first case. This means that we may forget details like primary intents, differentiations among EI,EO,EQ,ILF,EIF, number of DETs, RETs, FTRs in determining complexity of IFPUG BFCs. We have defined a new method using only two BFCs but we gave them weights in such a way that the translation rate between SiFP and IFPUG FP was exactly equals to 1. The paper will present the fundamentals of the method and some practical experiences of use.

Cigdem Gencel Blekinge Institute of Technology SWEDEN Cigdem Gencel is an assistant professor and a senior researcher at the School of Computing in Blekinge Institute of Technology, Sweden. She is a member of the Software Engineering Research Laboratory (SERL). Prior to this, she worked as a part-time instructor in the Informatics Institute of the Middle East Technical University in Turkey where she holds her PhD. Her research focuses on software measurement, software size and effort estimation, software project management, software process improvement, global software engineering, and software business value and quality management. Besides her academic career, she also has been working as a part-time consultant and trainer for industry on software size measurement, estimation and software process improvement since 2004. She has been involved in a number of industrial projects and she is a member of the COSMIC Metrics Practices Committee and International Advisory Committee.

Abstract: ‘Do We Really Need to Choose One Functional Size Measurement Method?’ SSimilar to any newly emerged engineering field, software engineering has been defined by the needs of the society looking for solutions to everyday life problems. Especially, the acknowledgment of empirical observation, experimentation and measurement as a pre-requisite for a rigorous theoretical foundation has accelerated the intellectual transformation in software engineering in the last fifteen years. Being one of the base measures, the size of the engineering products has received considerable attention of both researchers and industry. This has been also the case for software engineering with some other challenges due to the distinct nature of this new field. Various methods, techniques and approaches have been developed to measure the software product size, functional size being one of the widely and thoroughly researched one. Functional size measures the amount of functionality the software provides by measuring the aggregate amount of the cohesive execution sequences of the software. Each functional size measurement (FSM) method has its own definition of functionality and utilizes different abstractions during their measurement processes. Moreover, they use or look at the same concepts from different perspectives. Therefore, a software product would have different functional sizes in different units of measurement when measured by different methods. As a consequence, these differences resulted in some difficulties in practice such as how to select the most suitable FSM method for a software organization and how to make conversion between different measures in case of a need to compare the functional sizes of software products measured by different methods. In addition, the difficulty in defining a conversion formula creates challenges for establishing and utilizing benchmark sets. Therefore, it is not uncommon for software managers to encounter controversial statements regarding the merits of different FSM methods when they would like to choose the method for their software organization. Moreover, tremendous amount of effort have been put forth on independently to refine and extend the domain of application of each of – if someone may say- these competing FSM methods. On the other hand, looking at the base counts and the main measurement processes, FSM methods are actually based on similar concepts and make use of comparable attributes. Thus, there is a remarkable overlap among the methods, which actually points out a good agreement for what constitutes functional size. The differences are mainly due to different requirements of different functional domains when quantifying functionality and perspectives of different users. In this paper we elaborate on the conceptual similarities and differences between FSM methods and whether software organizations might benefit from all FSM methods without being obliged to choose ‘the one’ FSM method. This point of view might also open the door for joining the independent efforts of different FSM communities.

Frank Vogelezang Manager Pricing Office, Ordina (Nieuwegein) NETHERLANDS www.ordina.nl frank.vogelezang@ordina.nl Frank Vogelezang is currently managing the Pricing Office, which is the knowledge center for internal cost calculations for service offerings to customers. The Pricing Office is a long-term improvement program within the Proposal Management Center of Ordina. An operational task of the Pricing Office is to produce reference calculations based on models and experience data to challenge expert calculations. The Pricing Office also gives advice on the commercial pricing of service offerings. Before that, Frank has been working as a consultant on sizing and estimating IT projects since 1999. Frank is member of the Measurement Practices Committee of COSMIC and a member of the Counting Practices Committee of NESMA

Abstract: ‘Estimating and Pricing of Software Services: How a shared vision can improve the quality of service offerings' OOver the last decade Ordina has evolved from a constellation of specialised excellence centres to one of the larger local IT service providers in the Netherlands that is capable of executing large and complex service offerings. That evolution has impacted the way in which these service offerings are executed. In 2008 the division Application Outsourcing & Projects has been created to coordinate the service offerings and execution that require the expertise of more than one excellence centre. Estimating and pricing those kind of service offerings is more complex than service offerings for a single excellence centre, because of influencing activities where multiple excellence centres operate together. From a historical background, different excellence centres had different ways of estimating and pricing their service offerings. This complicated the overall estimating and pricing of large and complex service offerings. One of the measures for improvement was the start of the Pricing Office in 2010. The main objective of the Pricing Office is to improve the quality of the estimating and pricing of service offerings. This improvement program has three major areas of attention: the volume estimate (in man hours) of the offered solution the way in which the offered solution will be executed (staffing & sourcing) the pricing of the offered solution In order to communicate the way of working and the underlying principles a vision for estimating and pricing has been established. This model clearly communicates that there is a difference between estimating and pricing. Estimating is an engineering discipline that leads to the best estimation of the hours and associated cost of a combination of a solution for a contract and the way the contract will be staffed. Both the solution and the staffing can be influenced by commercial arguments, but the resulting cost price calculation is an engineering result. The quality of the cost price calculation has improved by the introduction of two tracks for the volume estimates (in hours) and the confrontation of both tracks to detect and mitigate as much risk in the estimate as possible. T The pricing process is essentially a commercial process to transform the cost price calculation to an offered price. The result of the pricing process is a pricing structure that is optimized to win the deal. An important input for this process is the set of acceptance criteria from the client. This pricing structure can be very dissimilar to the structure of the cost price structure. This model clearly states that there are different responsibilities in reaching an offered price: the engineers are in charge of the estimating process, the commercial departments are in charge of the pricing process. Abstract: ‘Estimating Application Management: Combining our Service Component Model with the estimating power of SEER IT' ‘Estimating Application Management’ Combining our Service Component Model with the estimating power of SEER IT Abstract: Although Application Management consumes 70-80% of the Total Cost of Ownership, estimating and bench¬marking the related effort still receives little attention. Apparently, estimating Application Management is not so simple. One of the main reasons for this, is the fact that the scope of Application Management is more diffuse than the scope of a project. When Application Management is outsourced to an IT service provider both parties need to have a clear understanding what activities will be outsourced and what activities will remain with the outsourcing organization. For the purpose of that understanding, Ordina has developed the Service Component Model as part of our Service Definition for Application Outsourcing. When we have defined what activities are in scope for an Application Management service offering, we are able to use this model as a basis to estimate the related effort and cost. To be able to do this in a transparent way, based on experience data, we have combined the vision of our Service Component Model with the estimating power and references of SEER IT. We have translated the Service Component Model to WBS elements that are predominantly based on the knowledge bases within SEER IT. We configured the WBS elements to be able to estimate Application Management for Oracle EBS. The configuration was calibrated using actual data from a number of contracts for Application Management in this domain. To validate the estimation power of this model we have re-estimated a number of service offerings with the knowledge we had at the time of making the proposal estimate. This model is now ready to be used as a reference estimate in the general approach for estimating and pricing of software services for Application Management in the Oracle EBS domain.

Clifford Shelley  Consulting Software Engineer,  Oxford Software Engineering. UK Clifford Shelley is a consulting software engineer and developer with OSEL, happy helping investigate and fix development problems, and sharing good practice. Previously a programmer (mathematical modelling and CASE tool development) in atomic energy and power industries. Later a software engineer and quality engineer with Hewlett Packard and then Racal Electronics. He was one of the first, outside US to use CMM, and has a subsequently helped many organizations improve and achieve recognition of their development capability at CMM and CMMI MLs 2 and 3. Clifford has a long standing interest in the software development process and in software measurement, having been both a victim and perpetrator of software measurement programmes. He is an experienced and respected trainer, a speaker at UK and European conferences, an active contributor and chairman of the British Software Process Improvement Network (SPIN), and a member of the UK Software Measurement Association’s committee. And a lapsed Scrum master.

Abstract: ‘Software Development Analytics’ The convergence of two software development trends, together with the recognition of some constraints on the use of measurement in software development presents an opportunity for software measurement to be rescued from its current state of disregard by the majority of the software community, and to develop as a widely applicable and valued tool. This presentation identifies the trends that can drive this potential recovery of software measurement, discusses some of the limitations of software measurement that have to be acknowledged if measurement and the information it provides are to gain wider acceptance. It also identifies proven but neglected analytical techniques that can be used to underpin this recovery. Sound analysis of software measurement data reveals and communicates the information it contains - leading to better understanding and decision making. Poor analysis may, and too frequently, does, lead to valued information being overlooked, or meaningless or spurious patterns emerging to mislead and distort decision making. This poor analysis is readily detected by others, even if the exact nature of its failings are difficult to pinpoint, undermining confidence in fundamentally sound data, and in software measurement in general. A widely applicable and robust approach is outlined as a framework for software data analysis. It is especially well suited to those non measurement specialists wishing to understand their complex or subtle software, software development (or other) data. The approach makes no assumptions about the nature of the data, beyond fundamental validation. It is scalable: analysis can be completed within minutes, or it can provide framework for a major investigation. It is progressive: with the analyst developing their understanding of the data as much as needed, and no more. It provides guidance on the selection and sequencing of analytical techniques, suggesting what to do (and what not to do) beginning with the most widely applicable, revealing and cost effective, and guiding analysts to more selective and specialist techniques as required. Because the approach is fundamentally simple and encourage good practice, as well as subtle and sophisticated, and because the techniques are not novel, but familiar and proven, the results will be accountable, communicable and credible, and an aid to shared understanding.

Keynote 3: Vince Groome Director of IT Solutions HM Revenue & Customs

Return to top

Keynote 4: Alain Abran Professor École de technologie supérieure Université du Québec CANADA Alain Abran is currently a professor and the director of the Software Engineering Research Laboratory at the ETS - University of Quebec, Canada. Alain Abran is an expert in his field, with two decades of experience of information systems development and software engineering and advanced degrees in engineering and management. Dr. Abran combines industrial experience and academic expertise. He is the co-inventor of COSMIC and the co-executive editor of the Guide to the Software Engineering Body of Knowledge, published by the IEEE. He is involved in standardizing software and system engineering (ISO/IEC JTC1 SC7). His recent book, Software Metrics and Software Metrology (Wiley IEEE Press) deals with the use of measurement to improve management practices in the context of software estimation and software quality. He is currently authoring another book to be published shortly, on COSMIC-based estimation. He has worked extensively on software productivity, estimation models, software engineering foundations, software quality, functional size measurement, risk management, and maintenance management, both in industry and with academics.

Peter Thomas Steria UK I am a Chartered IT Professional with experience in project management, design, implementation, test, and deployment of defence and commercial software. I am recognised as a technical specialist in software sizing, IT measurement, and estimating, in Steria, IBM, and the IT industry. I am an International Function Point User’s Group (IFPUG) Certified Function Point Specialist (CFPS). I was accredited by IBM to perform Quality Assurance. I am an experienced trainer and facilitator. I have experience of personnel selection, appraisal, and career counselling. I have been a member of the IFPUG Counting Practices Committee, “owner” of International Standards Organisation (ISO) standard ISO/IEC 20926 since January 2007. I was the technical lead of IBM’s Function Point Centre of Competence, which is the successor to Allan Albrecht’s IBM Research team which developed the Function Point software sizing technique. My role in Steria requires me to ensure productivity measurements are performed correctly, in particular Function Point counts to minimise contract penalties. Also I advise executives responsible for delivery on methods to manage the productivity performance against other targets including off shoring and contribute to bids. Steria is an IT Services company whose revenues have exceeded 1 billion Euros each year since 2005. My role in IBM involved business operations measurement coordination, quality assurance reviews, estimating support to bids, and Software Engineering Institute Capability Maturity Model Integrated (SEI CMMI) appraisals as well as occasional function point analysis to keep in contact with the real world of project delivery. Since 2004, I have regularly presented at technical conferences in the UK and USA and been invited to speak again as a result of the feedback. In 2006 & 2010 I presented the Ian Drummond Memorial lecture at the United Kingdom Software Metrics Association annual conference.

Abstract: ‘Defect Density and Project Profiling’ PPerformance measurement, for example to meet the needs of CMMi Organizational Process Performance, typically needs a quality /testing measurement. Many organisations measure defects. Some measure the ratio of defects to lines of code (whatever they are!!) This presentation shares the experience Steria has with this powerful measurement based on functional size both internally and for its customers. Attendees will gain insight that may provide the enabler they need to introduce functional sizing into their organisations.   Highlights:    • Definition of Defect Density (DD)    • Use of DD to the organisation    • Filtering incidents to identify defects    • Alternatives for size metrics    • Identifying other attributes and measurements needed for analysis

Brian Wells Experimentus UK Telno: +44 (0)7725 709262 email: brian.wells@experimentus.com Brian is a senior IT consultant at Experimentus Ltd specialising in Test & Validation. He has over 20 years experience within this sector and has held numerous senior IT positions including Test Strategist / Test Consultant within leading international organisations. Brian is a recognised thought leader within the Test & Validation industry and regularly speaks at industry events such as c testing solutions for clients. This includes defining, communicating and implementing corporate testing policy & strategy, processes & procedures, standards, test (design) techniques, metrics and monitoring programmes and measurement of Return on Investment (ROI), test tools, training, resources and organisation and much more. Brian has developed and implemented standards, processes and procedures (including test tools). As well as the definition and implementation of Metrics monitoring and a full Measurement system. He is a qualified CMM Assessor and TMMI Lead Assessor and has undertaken Test Maturity Assessments in many UK, Indian and other national and international Companies. In 2008 Brian was shortlisted for the Computer Weekly IT Professional of the Year award.

Abstract: ‘Successful Measurement of Test Process Improvement: Improve – Implement – Measure – Succeed!' To be more efficient, effective and competitive, organizations need to understand the capabilities of their development, test and quality processes. Using experience and analysis the presenter will start by reviewing the potential benefits of process improvement and specifically the benefits that can be obtained by reviewing and improving how we test using the TMMi model. Test Process improvement is a significant project in its own right and as such needs to be managed in a way that ensures that the right changes are made and that benefits (addressing visible and hidden costs of poor quality processes) are realized. Once you have identified what needs to change and how, how do you see that the changes are delivering benefit? You need to measure before during and after change is implemented! A cornerstone for effective change is measurement; process improvement and measurement are necessary bed-fellows! Often this is using the toolbox of measurement practices coupled to the requirement of the business to introduce a structured measurement programme across the organisation and assisting the organisation to effectively analyse and use this data for all dynamic and historical data analysis and presentation needs. During the presentation, we will look at how change and potential benefits of change were indentified within a leading Testing Services Provider that obtained TMMi Level 3 certification after a lengthy process improvement project (using a proven methodology to implement process change). We will also look at how targeted, pragmatic and appropriate measurements were implemented to demonstrate the value and benefits accruing (both tangible and intangible) in addition to the constraints evident within the particular service delivery context.

Eric Seufert Digital Chocolate ESTONIA Eric Seufert is an analytics engineer at Digital Chocolate, where he is responsible for all aspects of business strategy from pricing to customer engagement to software quality tracking. Until recently, he was the manager of Skype’s quality reporting program, where he handled quality analytics across Skype’s product catalogue from the formulation of metrics to report distribution. Prior to Skype, he worked as a strategy analyst at uShip.com, a venture capital-funded transportation start-up in Austin, Texas. Eric has an MA in Applied Economics from University College London, where he was an Erasmus Mundus scholar, and a BBA in Finance and Computer Science from the University of Texas at Austin.

Abstract: ‘Synthesizing Software Quality Metrics for Executive Operations Reviews’ This presentation presents an outline for communicating quality metrics at executive operations reviews. The presentation introduces the topic of software quality metrics and the most commonly used metrics in software quality reporting. It then illustrates the conceptual differences between normal engineering reporting practices and executive review reports as they pertain to quality metrics, defining a set of metrics that can be used specifically for executive reviews. The remainder of the presentation describes a framework for interpreting software quality metrics in developing product strategy.

Thomas Fehlmann Euro Project Office Switzerland Dr. Thomas Fehlmann obtained in 1981 a Ph.D. in mathematics from the Federal Institute of Technology in Zurich (ETH-Z). He became a "Black Belt" for Six Sigma for Software implementation in 1991. Since then, he led Six Sigma improvement projects and made various contributions to the development of quality management and techniques especially for software development and systems integration. In 1999, he started Euro Project Office to support customers with Six Sigma techniques and projects. Shortly thereafter he received the renowned according the criteria of the German QFD Institute. Combinatory Metrics for software product management were first published 2004 in the Emerald Journal for Reliability Management. Since 2003, he is SW Metrics expert in SwissICT, the largest organization of Swiss ICT professionals, and Swiss delegate to ISBSG, the COSMIC group, and MAIN, the Metrics Association International Network. He is a frequent writer, facilitator, teacher, and counsels customers in Europe, Asia, and the Americas.

Abstract: Measuring and Estimating Ongoing Agile Projects in Real-Time Using COSMIC, it is possible to maintain sizing information while an agile software development project evolves. Based on UML sequence diagrams in User Stories, functional sizing counts can be maintained from the beginning until the end. However, for estimates this is not good enough. While early estimates based on functional size and cost drivers make sense in the inception phase, during development, the Buglione-Trudel Matrix provide much better value to the development team and the sponsor. Based on contributions to the business drivers of the project, actual estimates can be updated and maintained in real-time at least whenever starting a new sprint. Sponsors can assess how long the project will continue, and the team can rely on benchmarks for effort estimation, rather than on planning poker. This paper highlights the problems actually encountered while measuring progress and direction of agile software development projects.

Grant Rule SMS Exemplar (Group) Ltd UK paper presented by Chris Woodward in commemoration of Grant who passed away Our gratitude goes to Grant and his family. Grant Rule is co-founder of the UK Rightshifting Network, an Executive Coach for Orchestrated Knowledge & MD of SMS. He has 39+ years experience in ICT. Public and private sector organisations engage Grant to facilitate learning and increase effectiveness of their value creation processes. As 'Quantity Surveyors of Information Systems', Grant and his colleagues work with decision makers to maximise the value of business projects that rely on software. They recognise that decision-makers need predictability, value-for-money, short demand-to-delivery times, plus acceptable product quality to build & sustain customer loyalty. To reduce the waste commonly found throughout the software life-cycle, Grant has pioneered the application of lean systems thinking & agile practices since 2002. This is a natural extension of his experiences in the mid-1970s using Chief Programmer Teams, rapid iterations (‘single useful next step’), deferred commitment, common ownership of work-products, co-location of stakeholders, and techniques now called ‘agile’. In the 1990s, with Ken Dymond he introduced public training in the CMM® into England, and brought to Europe the first public ‘Introduction to the CMMI®’. He worked with Kent Johnson to align CMMI®-based improvements with lean-agile methods including Scrum, XP and Kanban. Grant served on the Advisory Board to IEEE Software for ten years to 2007.

Abstract: Predictable Pricing for Agile Development Customers are risk-averse. They value assurance and predictability. As a result, they want to stipulate detailed specifications early in the procurement process. Inevitably, such detailed specifications have a very limited ‘shelf-life’, are incomplete, inconsistent and otherwise ill-informed. Both parties end up stuck with contract terms and performance measures that act as barriers to a constructive collaborative dialogue focused on value delivery. Agile developers address this issue by offering an iterative development process which recognises that requirements cannot be defined up-front. But for those procuring outsourced services, such an approach can seem risky - what assurance does the customer have that promised benefits will be realised? How can you measure and compare the efficiency of agile delivery, and ensure you are paying a reasonable price for an effective service? How does the customer retain control of what is delivered? How do they budget for agile delivery in a way that does not detract from the value delivered? In this presentation, Grant Rule shows how contracting with COSMIC provides the customer with the assurance they need - predictable costs and value for money – and leaves the supplier the freedom to apply their specialist know-how to designing a solution and delivering a profitable, quality service at a competitive price. COSMIC gives clients and suppliers an objective measure of scope on which to negotiate. It enables suppliers to quote a competitive unit price which can be instantly benchmarked against industry averages. Smaller software houses can instantly demonstrate competitive levels of productivity compared to larger suppliers.

Carl Bideau MA (Cantab) Capgemini UK Carl Bideau has worked in the IT industry since 1985 in a variety of roles including analysis, testing and project management. He has been a member of the Capgemini Estimation & Measurement Centre since its inception in 2000. He has extensive experience in bottom-up & top-down project estimation, post-project metrics collection/analysis, productivity measurement (baselining & benchmarking) and project progress tracking for custom software development and other project types (application integration, data migration, large scale package implementation etc.) and is involved in leading the Capgemini global approaches in these areas. Carl is certified by the Open Group as a Master level Certified IT Specialist:

Abstract: ‘Application development Productivity Measurement - some Concepts & Considerations’ This presentation discusses the various considerations, factors and concepts that need to be taken into account in order to be able to sensibly measure and interpret application development productivity. A distinction is made between “product size” and “project-specific productivity factors” and non-functional requirements are also addressed. The main focus is on custom software development (CSD) although mention will be made of other project types. Measurement of application support is beyond the scope of this presentation

Ozden Ozcan Top Middle East Technical University Ankara, Turkey ozden@ii.metu.edu.tr Mina Nabi Middle East Technical University Ankara, Turkey mina@ii.metu.edu.tr Onur Demirors Middle East Technical University Ankara, Turkey demirors@metu.edu.tr

Özden Özcan Top is a PhD student in Middle East Technical University, Informatics Institute, Information Systems program. She has an MSc degree in the same program, same university and has a BSc.degree in Industrial Engineering, Yildiz Technical University. She is currently working as a software quality specialist in FUJITSU Company. Her research interests include; software quality, process improvement, software size measurement, effort prediction, software benchmarking and software project management. Mina Nabi is an MSc student in Middle East Technical University, Informatics Institute, Department of Information Systems. She has B.S degree from Computer engineering Department in software engineering program, Tabriz Azad University. She worked as senior programmer in a software consultant company in number of projects on ERP (enterprise resource planning) systems. Her research interests include; software measurement, software estimation, and software project management. Onur Demirors is an Associate Professor and the Chair of the Software Management Programme of Information Systems Department at Middle East Technical University. He is also the strategy director of Bilgi Grubu Ltd. He holds a Ph.D. degree in Computer Science from Southern Methodist University. His research focuses on software process improvement, software project management, software measurement, software engineering education, software engineering standards, and organizational change management. He managed research projects on developing software process improvement and modeling techniques for SMEs and for establishing and implementing business process modeling approaches for large scale software intensive system specification/acquisition. His work on software measurement particularly focuses on development of new functional size measurement methods and effort estimation techniques as well as establishing benchmarking datasets and validating current methods with field experiments. He worked as a consultant for a number of software developing companies to improve their processes based on ISO 9001, ISO 15504, CMM and CMMi. He studies business process management for software organizations from measurement and modeling perspectives as well as process mining techniques.

Abstract: ‘Comparison of Software Benchmarking Repositories from Effort Prediction Perspective’ Benchmarking repositories play critical role in software management practices. Construction of the estimation models, comparison of software performance indicators, identification of the process improvement opportunities, and quality assessment are major utilization areas of software benchmarking repositories. This paper presents the results of a comparative study on the external benchmarking repositories from effort prediction model construction perspective. ISBSG, IPA/SEC, CSBSG and other 11 data repositories developed by researchers for empirical studies were selected for comparison. The approach for evaluation was the assessment of the data repositories’ capabilities based on predefined criteria. This study has revealed that data repositories are not easily usable for software organizations to establish their effort estimation models. The significant problems include issues related with data validation; insufficiency of data for large scale projects; repositories’ lack of detailed effort data and some essential attributes for effort estimation. Lack of a benchmarking standard and a support tool for data collection are the other major causes for problems. Therefore data repositories should be improved from data collection, validation and maintenance perspectives for more accurate effort estimation models.

Ton Dekkers Galorath International UK/Netherlands Ton Dekkers is working as a practitioner, consultant, manager and trainer within the area of project support, software measurement and quality assurance for over 20 years. Within these areas he specialised in estimating, performance measurement (Sizing, Goal-Question-Metric, Benchmarking), risk analysis and scope management. His current position is Director of Consulting for Galorath International. In addition to his regular job he is Immediate Past President of International Software Benchmarking Standards Group (ISBSG), President of the Netherlands Software Measurement Association (NESMA), member of the International Advisory Committee of COSMIC and member of the SIG parametric estimation of the Dutch Association of Cost Engineers.

Abstract: 'Software Estimation Maturity' In the CHAOS Standish report the estimation (process) is mentioned as a potential cause for challenges in a project of program. A wrong estimate is most of the time double-trouble, challenges are seen in a financial context (cost) and in a delivery context (duration, effort). Over the last two decades a lot of improvements are made. This is reflected in ISO standards for sizing (IFPUG, NeSMA, COSMIC, Mark II and FiSMA) and benchmarking initiatives (e.g. ISBSG – International Software Benchmarking Standards Group). However, looking at other industries, the IT industry doesn’t have implemented the basic Cost Engineering principles as part of their standard way of working. You can find depending on personal interest, market position or services portfolio, elements of Cost Engineering implemented. In the market is quite some interest in improving the estimating practices. One of the challenges is the definition of such an improvement. To support the decisions on this matter, the Software Estimation Maturity Model is defined. In conjunction with the model, an assessment structure is developed. This enables to assess the level of “maturity” of the organisation and the current practices. In addition the model can be used to define which level of maturity is required for the organisation. Both, the maturity model and the (self) assessment questionnaire are based on lots of lessons learned when implementing estimation processes, measurement programs and project management offices. Because of this knowledge the required level of maturity for the organisation can be defined and related to that an implementation / improvement plan can be created. Best practices are reflected in the models and will speed up the improvement process. In addition a link will be made with the implementation approach MOUSE, that initially was developed for the implementation of a Functional Size Measurement Method.. The generic principles are useful for describing the setup of the organisation, the activation and motivation of participants and finally the support.

Return to top