J. Sugden, Pickering, UK
The paper starts from the proposition that the choice between highway design options depends on ‘intuitive judgements, sometimes without utilisation of clear criteria, a focused approach and/or systematic mathematical processes’. The conclusion from this should be that there is a need to develop a clear and transparent decision system that reflects the needs of the public.
However, there is also a less palatable implication: that the lack of a proper system in the past will have led to non-optimum solutions, not only because of lack of consistency but also because engineers and decision-makers have adopted intuitive criteria that did not correspond with public need. It is by no means inconceivable that professionals involved in the design of trunk roads will tend to give undue prominence to the needs of long-distance traffic in comparison with, say, the local environment or the needs of local non-motorised user movements.
For this reason I would argue that the whole basis of the paper is flawed, because it is based on an analysis of how engineers and decision-makers currently come to decisions. This means that any unconscious bias present in the current intuitive approach will be carried forward to the new. The authors claim that the methodology has been successfully used, but there seems no way of demonstrating success. Any optimising model may be expected to indicate an optimum solution, but it is by no means obvious that this will actually give the right answer.
A further flaw appears to be an assumption that all criteria may be subject to trade-offs. In practice, many criteria consist of fixed constraints, either derived from statute or relating to public perceptions of equity. For example, avoiding important conservation sites and the replacement of new habitats for those lost should be regarded as constraints. Similarly, the needs of NMUs should also be taken as constraints, with particular emphasis on the needs of people with disabilities.
I would submit that the correct solution to the initial problem is the use of cost–benefit analysis, as has been the core of evaluation since the 1960s. The developments required are a clear identification of the design constraints as discussed above, and the monetisation of the relatively few aspects that are neither subject to constraints nor already monetised. Possibly the most important is the valuation of the benefits to property owners of reduced traffic arising from new roads. Given the British public's obsession with house values it is difficult to see that this would present a major problem.
An approach based on these principles would lead to a system of evaluation of options that reflected what the public want, rather than the possibly biased views of engineers and decision-makers.
Authors' reply
The paper's main focus is to show the advantage of using the analytical hierarchy process (AHP) in helping to model a complex decision situation. The hierarchical model is constructed based on three levels: criteria, subcriteria, and alternatives. We agree that design engineers will be inherently biased. However, the difference between this process and other decision-making tools is that it incorporates the needs, the objectives and the bias of other stakeholders at the onset of the project. This is balanced by additional stakeholders with conflicting needs and objectives, both of which are negotiated during the development stage of the decision model, thus enabling the adoption of an impartial system. In our decision model the stakeholders' needs and objectives have been incorporated between the subcriteria (Level 3) and the alternatives (Level 4). Professor Saaty's book on AHP (reference 2 in our paper) addresses the issue of including stakeholders' views extensively.
‘Success’ is also defined at the onset of the project, and in this instance is considered to be a solution that has the highest score and fulfils the criteria and relative priorities of the agreed system. Design is a process of compromise: that is, the best compromise, not necessarily optimisation. We would therefore argue that the ‘right answer’ is a solution that is acceptable to all parties.
Regarding the assumption that all criteria may be subject to trade-offs, and that in practice many criteria consist of fixed constraints, we argue that in this instance, where there are constraints, these are incorporated into the design (not the scoring process), and therefore all comparative designs would have the same constraints. The aim of the scoring model is to identify those areas that can be modified in the design and provide a rational means of assessing their relative benefits or merits.
‘Costing the benefits to the public’ is a longer and possibly an even more subjective process. The advantage of the AHP process is that it enables the user(s) to make informed decisions by considering both subjective judgements (benefits or disbenefits) and disparate criteria that may or may not have a monetary or other value.
Although cost–benefit analysis has been used extensively in the past by many practitioners and government officials, it has been recognised that in some instances it is impractical to define monetary values for all major costs and benefits. In the year 2000 the Department for Transport, Local Government and the Regions (now known as the Department for Communities and Local Government) published the multi-criteria analysis (MCA) manual, which can be downloaded at
http://www.communities.gov.uk/pub/252/MulticriteriaanalysismanualPDF1380Kb_id1142252.pdf (last accessed December 2006)
The manual describes the rationale, merits and limitations of the suite of MCA techniques, with AHP being one of them, and encourages practitioners to use these methods as alternatives to monetary valuations (i.e. financial analysis, cost-effectiveness analysis and cost–benefit analysis) for the appraisal of options for transport projects.
