56. I do not feel comfortable with this report in some ways. Its methodology is really an opinion survey of a number of people chosen we do not know quite how. I am not sure that measuring perceptions tells us very much, especially when you are asking people, "Are you in favour of motherhood or apple pie?" Underlining some of my questions will be that general theme, that we have here an opinion survey rather than any objective study of value for money or anything else, although there is some interesting information coming out of it. Sir John, are you in favour of motherhood and apple pie? It seems to me that the culture of this place now is that PFI is the latest flavour of the month. Personally I am not particularly in favour of it but I guess if you asked the majority of civil servants, "Are you in favour of PFI and do you get value for money?", it is a bit like saying, "Are you in favour of motherhood and apple pie?"
(Sir John Bourn) Yes, but that of course was not what the survey was about. The survey was about the experience that people had of working the PFI contracts and the experience showed that there was a degree of satisfaction. The experience also showed that satisfaction was declining. The experience also showed that on both sides people had ideas on how to make things better. It was-and I do not claim for it any more than it was-a survey of those people who were responsible for this work and a survey of what their experience was. As Mr Gershon has said, this is the first time that an attempt has been made to provide information on this area at all and that does not mean to say that we will not do further work which examines this from other bases.
57. I have no doubt that there is useful information in here but you really are asking people about perceptions: "Do you perceive that you have had value for money?", and then we have made a table which members have made much of about whether the people who are managing accounts perceive that they have achieved value for money. I guess most people are going to say yes, and in a way it is a self-fulfilling approach. That is my view and I just want to say that some of the people of whom we have asked questions are clearly not equipped to give answers. I will demonstrate that in the questions which I am going to address to yourself and others. First of all can I ask on what basis you selected the projects? There were 400 projects altogether. You chose 121 of them. Actually you did not even manage to get some of them to reply, did you?
(Mr Finlay) The sample that we chose was virtually all of central government PFI contracts.
58. Every one of them?
(Mr Finlay) At the end of 2000 which had been in force for over a year. That was the basis of the sample.
59. Not everybody replied though, did they?
(Mr Finlay) Not everyone replied but we did get a high proportion who replied. What I would also say is that although you have mentioned that we asked perceptions about value for money, which is absolutely true, we also asked a series of much more detailed questions to build up a very close understanding of the mechanics of what was going on.
60. We are politicians and we try to understand opinion polls and that is what this really is. We also know that opinion polls depend on accurate sampling. When you get a rogue poll it is because the sample of population that was asked was not truly random. We have only got a limited number of responses. Ninety per cent of the authorities but only about three-quarters of the contractors bothered to respond. If you look at the detail of the report many of them did not fill all the questions in, did they? They did not answer all the questions.
(Mr Finlay) Certainly in the answers on value for money we got a high proportion of responses there.
61. What steps did you take to find out whether the people who did not respond were not skewed in some way? How do you know that the people who did not respond were the ones who were most disgruntled, unhappy or confused?
(Mr Finlay) We certainly took steps to get as high a response rate as possible.
62. Yes, but I did not ask you that question, did I? How do you know that the ones who did not respond were not the examples of worst practice?
(Mr Finlay) Clearly if we have not got the response from an authority or a contractor then we do not know exactly what their view of the contract is. What I would say is that having devised the survey by reference to, as I say, virtually all central government PFI contracts which had been in force for over a year at the end of 2000, it was a virtually comprehensive sample that we started from and the list of projects at the back does show a very wide ranging list.3
63. It does. I have made the point, which you have not responded to but it is on the record because I have made it, that even the ones who responded did not reply to every question. In some cases your analysis was only based on 72 responses, in table 12, for example. If we turn to the outcomes of this opinion poll, I am surprised that anybody says we have not achieved value for money because presumably they are required by their contracts of employment to produce value for money. In a way it is back to this motherhood and apple pie thing. When we looked at the skill levels of the staff who were managing the contract I thought that that was useful information. I think it is very valuable to us. I will just turn to Mr Gershon. Are you happy with the information which was revealed, for example, in table 12, admittedly on a relatively small sample, which indicates that significant numbers of the staff were not appropriately skilled or experienced in managing contracts, does it not?
(Mr Gershon) The contract management team should have had experience in the appropriate skills. I am happy with that but there is some cause for concern with these figures. In any team you would expect to see a blend of experience and appropriate skills. Otherwise how will young people ever become the experienced people of the future? Where you see the percentages below 50 per cent, that would be cause for concern.
64. Seventeen out of 72 were precisely less than 50 per cent.
(Mr Gershon) Yes.
65. So in 17 out of 72 cases, according to your answer just now, and this is design and build, you are unhappy with the blend of skills there.
(Mr Gershon) Yes, I am.
66. Then we note elsewhere that the number of staff who were then transferred from having designed the contract, prepared it and presumably let the contract to managing the contract, it is fascinating that significant proportions of contracts were managed by yet another group of staff who had not been involved in the preparation of the contract. Are you content with that?
(Mr Gershon) No, I am not. It is because of that that we recognised that that was a problem some time ago, which is why in the Gateway review one of the things that is explicitly tested is things like the skills and capabilities of the team at the design and build stage and that the skills and capabilities of the team at the procurement stage, and we also look at what are the plans for contract management and test what plans the department has for an acceptable level of continuity.
67. Two-thirds of the contracts were managed by staff who had no or little involvement in the procurement of the contract.
(Mr Gershon) Incidentally, we may not like these results but it does illustrate that people who answered the survey were pretty honest.
68. I am going to come back to the C&AG in a moment because this question reveals two things: firstly, contract preparation and contract management are conceptually separate from a civil servant's point of view which I would have thought you would be unhappy with; secondly, the people who are preparing contracts are not all equipped to prepare those contracts in many cases. The question then is, if that is the case, were these the same people who replied to the questionnaire because we are compounding the problem of levels of competence if we are then asking these less than competent people to respond to say whether they felt that the contract was well managed? I think it rather proves the reverse of what you were saying, does it not? Let us just stick to the question which is about levels of competence in terms of contract management and contract preparation. Are you happy with the information that has come out?
(Mr Gershon) I have just said that I was not. Ideally I should like to have seen no entries in the 0-25 per cent and the 26-50 per cent ranges. Mr Williams always thinks we are a bit slow but in this area with the Gateway review process, from when we first launched it last February, we have been helping the department take a whole life approach to these forms of complex projects, not just PFI contracts but other complex projects as well, where we test explicitly for the skills in the public sector team in the procurement phase and the plans for the skills in the public and private sector teams in the design and build stage and the service provision stage. We highlight to the Accounting Officer where those reviews identify the sorts of weaknesses that this report is also highlighting.
69. Let me go on to table 8. I may not have time to get back to the C&AG. These are supposedly the ways in which VFM was guaranteed objectively according to the perceptions of the people responding to the opinion poll. It is interesting to see that large numbers of contracts do not seem to have had VFM exercises done on them, objective tests. If you take bench marking, for example, which I am not totally convinced is an accurate way of securing value for money, how convinced are you that the staff who were doing the bench marking were competent?
(Mr Gershon) Bench marking as far as I am aware not only involves the authority's own staff. They may well bring in third parties who have specialist market knowledge to assist in the bench marking exercise.
70. But it may well involving bringing in some extra staff.
(Mr Gershon) There are examples that I am aware of where I know that that has been done.
71. What guidance are you giving to the authorities and the departments as to these types of methodologies for securing VFM? Are you happy with these statistics?
(Mr Gershon) No, I cannot say I am happy because to say 15 per cent shared in refinancing benefits is cause for concern. I have indicated already through the revised advice and guidance we are giving departments that we have to take a much stronger position in the sharing of those sorts of benefits. The direct answer to your question is no, I am not happy with those percentages and we are doing something about it.
72. Okay. If I can just come back to the C&AG because I have been given my short term warning that time is running out. Mr Gershon is not too happy with the level of competence that has been revealed in terms of contract management and the procuring of contracts, the skill of those people. How happy are you about the level of competence of the staff who fill in your questionnaire? What steps did you take to ensure that these people were competent? I would guess that if it landed on my desk and I was the chief officer in the department I would pass it down to the person who was doing the contract to fill it in and probably make sure the boxes were filled in before it came back to yourself. Is that apple pie and motherhood and things really? What is this measuring, this document, in a way?
(Mr Finlay) In terms of the people who filled out our questionnaires, they were sent to the people currently in charge of the projects, whether that is on the authority side or the contractor side. Certainly from a lot of the feedback that we got, because we were regularly in touch with the people who were completing these questionnaires, they were giving it a lot of consideration and the people at the top of the teams, even if they delegated part of the exercise to people beneath, the people at the top of the team were taking responsibility for sending it back to advisers.
73. Many of the people who were managing the contracts had not actually prepared the contracts in the first place. Let me just say this-I have to finish- you were asking them had their perception of the value for money changed and yet many of the people were either not qualified to manage the contract and/or had not even prepared the contracts in the first place.
(Sir John Bourn) If they had not prepared the contract in the first place they were not parti pris to it and were not under any obligation of previous activity to say what they did not think. It was an opportunity for them to say what they really thought.
Jon Trickett: In trying to demonstrate the dynamics of perception as they change over the lifetime of a contract, we now know from this report actually that the staff were changing. You were asking the staff at the time at which the survey went out rather than the time it was submitted. I have now received my final warning.
Chairman: You can still get an answer.
Jon Trickett: No, I would like to get the last word in.
__________________________________________________________________________________