Our review of nine contracts also found that the contract managers rarely had documentation for their acceptance of deliverables. In fact, the contract managers for seven of the nine contracts did not have any acceptance notices on file. Authority policies and procedures require contract managers to review deliverables for compliance with contract requirements and the Authority's acceptance criteria. The contract managers must then officially accept the deliverables in writing. This documented communication marks the point in time when the Authority takes care, custody, and control of the deliverable.
Despite this requirement, when we asked the contract manager for an environmental contract about the missing acceptance notices, he stated that the contract had only two real deliverables, one of which is the environmental impact report. He stated that he planned to evaluate that report upon its completion and that he would issue the acceptance notice at that time. However, the contract documentation makes clear that the contractor is responsible for many individual work products as part of developing this larger report. When we asked about these work products, the contract manager stated that instead of reviewing deliverables as the contractor submits them, he plans to later issue a memorandum to summarize all deliverables. We received similar responses from other contract managers, who stated that acceptance notices were not applicable because the contracts they managed did not include what they considered to be formal deliverables. However, we disagree with this assertion; for each of these contracts, we identified evidence of discrete work products-or deliverables-that the Authority policies and procedures would require contract managers to evaluate and formally accept.
In fact, only two of the nine contracts had acceptance notices on file that documented the timing of deliverable submission and review, and this documentation was minimal and used inconsistently. When we requested acceptance notices for one of the two contracts-a four-year, $40 million contract for financial advising services-the contract manager provided nine notices, all of which he signed and dated June 11, 2018-after the date of our request on June 4. Further, none of the notices contained any comments in the template fields that contract managers can use to record the details of their reviews. When we subsequently followed up to ask about additional deliverables for which the contract manager had not provided acceptance notices, he produced another three notices, each of which he signed on July 9, 2018-again, after our request. In some instances, the contractor had submitted the deliverables in question as early as October 2017. Because the Authority pays the contractor for its work on a monthly basis, even if the contract manager had performed a detailed review in June and July 2018 and had found issues with the deliverables, the Authority had already long since paid the contractor for the work. | Contrary to Authority policies and procedures, only two of the nine contracts had acceptance notices on file that documented the timing of deliverable submission and review, and this documentation was minimal and used inconsistently. |
When we asked the contract manager about his signing the nine notices on the same day, he asserted that the process was for the contractor to submit the acceptance notices to him at the end of the fiscal year, rather than with the deliverables. However, as we state above, he provided the original set of acceptance notices before the end of the fiscal year, and soon after we requested them. Further, because the acceptance notices are a part of the Authority's policies and procedures, we would expect the contract manager to initiate the process, not the contractor. When we expressed concern about the contract manager's approach, he replied that he believed the process was sufficient because he would complete the notices before the contract was closed. However, if the Authority waits until the contract's end to review deliverables, it will have already substantially paid for these deliverables. When we asked about any other evidence to demonstrate his review of deliverables, the contract manager provided emails intended to demonstrate his review. Our review of those emails found that the contract manager was not included on many of them, some of which were between contractors only. The contract manager told us that going forward, he plans to complete an acceptance notice contemporaneously with receipt of each deliverable. To the extent that the contract manager also performs and documents his comprehensive review of each deliverable at this time, we agree it is appropriate for him to complete the process by signing and issuing acceptance notices.
We also identified significant concerns with the Authority's collection and evaluation of deliverables for its $666 million RDP consulting contract, which includes tasks in 30 subject matter areas. In late 2017, the contract manager tasked a contractor-from a firm other than the RDP consultants-with conducting an assessment of the deliverables and corresponding acceptance notices that the Authority had on file dating back to the start of the RDP contract in July 2015. To conduct this assessment, the contractor compared the RDP consultants' self-reported information about the deliverables it had submitted to the Authority to the deliverables the Authority actually had on file. The review determined that for the work plans that were active when the Authority's contract management policies and procedures went into effect, the Authority was missing formal acceptance notices for 70 of the 80 deliverables that the RDP consultants reported as complete. Further, the Authority did not have 145 of the 184 deliverables that the RDP consultants had reported as having completed since the beginning of the contract. These missing deliverables ranged from engineering documents to software updates to white papers and other strategic documents. Nonetheless, from July 2015 through December 2017, the Authority paid the RDP consultants over $200 million for the tasks that included these deliverables. | The Authority was missing formal acceptance notices for 70 of the 80 deliverables and did not have 145 of the 184 deliverables that the RDP consultants had reported as having completed since the beginning of the contract in July 2015. Nonetheless, from July 2015 through December 2017, the Authority paid the RDP consultants over $200 million for the tasks that included these deliverables. |
When we asked about the Authority's efforts to follow up on the missing and unapproved deliverables, the contract manager told us that because of funding constraints, the non-RDP contractor was no longer working on the deliverables issue. However, he stated that he had requested help from other Authority staff in identifying and recovering the deliverables. Nonetheless, as of July 2018, he had not yet received any new information. Without documentation of formal review and approval, the Authority cannot demonstrate that it received the quantity and quality of work for which it paid the RDP consultants.
Moreover, the Authority's tracking and evaluation of this contract's deliverables has continued to be an issue. In July 2018, the contract manager provided us a list of deliverables that the RDP consultants had reported submitting, which he originally represented as an updated tracking log. However, the contract manager later stated that because of the aforementioned funding constraints and a lack of support staff, he had not had the opportunity to verify the submission and timing of the deliverables on this list. He asserted that subject matter experts, who are state employees, are involved in developing the monthly status reports that the RDP consultants submit to the Authority, which include the status of deliverables. However, he acknowledged at the time that he had not reached out to subject matter experts to collect the deliverables that the RDP consultants had reported as complete and therefore had not yet completed deliverable reviews in order to issue acceptance notices. Despite having confirmed this lack of acceptance notices on multiple occasions, the contract manager informed us in October 2018 that he did in fact have completed acceptance notices for some of the RDP consultants' deliverables. He then provided 77 signed acceptance notices dated as far back as January 2018.
These acceptance notices do not alleviate the need for detailed deliverable tracking and review documentation. The acceptance notices on their own do not allow the contract manager to determine whether the RDP consultants' work is generally on schedule. Along with the acceptance notices, the contract manager also provided a log his staff began compiling in August 2018 to track the status of deliverable acceptance. If used going forward, the log will help the contract manager more proactively track the status of all deliverables. However, even though the contract manager asserted that the acceptance notices are the formal deliverable review documents, neither the acceptance notices that he provided nor the log contain detail about how the contract manager determined that deliverables met contract requirements. | Even though the contract manager asserted that the acceptance notices are the formal deliverable review documents, neither the acceptance notices that he provided nor the log contain detail about how the contract manager determined that deliverables met contract requirements. |
Given the large dollar amount associated with this contract, it is crucial that the Authority improve its practices for tracking the status of deliverables and reviewing them for quality.