Audit Report
November 2018
Project Number: 80590-107
- Executive summary
- Conformance with professional standards
- Introduction
- Background
- Audit objectives
- Evaluation objective
- Scope
- Approach and methodology
- Authority
- Findings, recommendations and management response
- The client's perspective
- Governance and risk management
- Processes and controls
- Appendices
- Appendix A: Audit criteria
- Appendix B: Initialisms
- Appendix C: Sampling methodology
- Appendix D: Client satisfaction with cost-recovery processes (select aspects)
Executive summary
Statistics Canada's mandate is to fulfill the information needs of governments, businesses and individuals for trusted data that will help improve the well-being of Canadians. Statistics Canada (the "Agency") delivers its mandate through its core program and by meeting specific information needs through optional cost-recovered statistical services work.
Statistics Canada is considered a common service organization for optional services under the Treasury Board of Canada (TB) Common Services Policy. The Agency is authorized to provide optional statistical services to other federal departments and agencies and to offer these same services to Crown corporations or non-federal organizations. These optional services are to be provided on a contractual cost-recovery basis, and rates charged are to recover but not exceed full cost of the services provided.
In 2017/2018, the Agency conducted approximately 300 statistical surveys and related services projects, and a large number of custom requests and workshops. Statistics Canada's full cost in providing these cost-recovery activities was approximately $120 million, which was recovered through contractual arrangements with service recipients (i.e., "clients").
Statistics Canada undergoes scrutiny and is challenged by clients in relation to the cost of statistical services. Given this, the appropriateness of the Agency's costing methodology and approach to estimating project costs for statistical services has been questioned periodically.
Statistics Canada continues to strategically position its cost-recovered statistical services to ensure its sustainability by providing high-quality products and services that add value. Operational effectiveness, however, is predicated on having a cost-recovery governance framework in place that promotes responsive client services and relevant information gathering and analysis. In relation to this, pricing to clients must be based on cost estimates that accurately reflect the Agency's full cost of providing statistical services. To this end, effective protocols, procedures and processes must be in place.
Why is this important?
Statistics Canada conducts a high volume of cost-recovery activities. Public profile of this client-facing program is high, and demand is increasing in light of the government's commitment to evidence-based policy and programming. Given the materiality, scope and public nature of the Agency's cost-recovered activities, combined with the increasing scrutiny of costs by its clients, it is essential that the Agency ensure effective management of this highly visible and heavily relied-upon program. This audit supports this goal and contributes to the Agency's modernization agenda by providing important insights directed at improving the operational, financial and administrative controls and processes associated with this program.
Key findings
Clients are very positive towards many aspects of the Agency's cost-recovery processes, and most clients intend to continue using Statistics Canada's statistical services. Clients generally choose Statistics Canada as a service provider for data quality and reputation reasons.
Two areas for client satisfaction improvement were identified: (1) clarity and transparency of costs; and (2) access to data prior to release.
Roles and responsibilities have been clearly defined and communicated. Responsibilities for the delivery and financial management oversight of cost-recovery projects are shared and managed jointly at various levels across the Agency.
The quarterly program financial review process is the key control to validate the quality of project cost estimates. The review process allows Statistics Canada to monitor cost-recovery project costs to budget and know whether a particular project is forecasting a surplus or deficit.
Formal, Agency-wide risk management processes are in place, though there is no Agency-wide mechanism to consolidate, store and analyze information on trends and changes in cost-recovery activities during a given fiscal year to help ensure that Statistics Canada remains relevant in meeting evolving customer needs.
Statistics Canada has developed Agency-wide standard costing rates that are reviewed regularly and approved by the Corporate Planning Committee on a yearly basis. The standard costing rates fairly reflect the Agency's relevant costs.
Common tools and templates are consistently used to develop project cost estimates and enter into contracts with clients. The Agency works collectively to deliver project results to clients at the agreed upon price, and material project level surpluses or deficits are rare.
The effectiveness of quarterly program financial reviews could be negatively impacted by some existing time coding practices.
Overall conclusion
Statistics Canada has developed an effective governance framework to manage its cost-recovery activities, and effective processes and controls are in place. The Agency's costing methodology is well documented in the Statistics Canada Costing Guideline, and common costing and contracting tools and templates are consistently used. Clients are also very positive towards many aspects of the Agency's cost-recovery processes, though areas for process and client satisfaction improvement were nonetheless identified.
The Agency can better support risk management and decision making by improving processes to capture and report on business and market intelligence. Changes to time coding practices can also help the Agency to more accurately identify risks, best practices and areas of improvement. Finally, Statistics Canada can improve the clarity and transparency of its costing methodology and can further improve client satisfaction by exploring ways to address concerns about access to data prior to release.
Conformance with professional standards
The audit was conducted in accordance with the Mandatory Procedures for Internal Auditing in the Government of Canada, which include the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing.
Sufficient and appropriate audit procedures have been conducted, and evidence has been gathered to support the accuracy of the findings and conclusions in this report, and to provide an audit level of assurance. The findings and conclusions are based on a comparison of the conditions as they existed at the time, against pre-established audit criteria. The findings and conclusions are applicable to the entity examined, and for the scope and period covered by the audit.
Steven McRoberts
Chief Audit and Evaluation Executive
Introduction
Background
Statistics Canada's mandate is to fulfill the information needs of governments, businesses and individuals for trusted data that will help improve the well-being of Canadians. Statistics Canada (the "Agency") delivers its mandate through its core program and by meeting specific information needs through optional cost-recovered statistical services work.
Cost-recovered statistical services are part of the Agency's Cost-recovered Statistical Services Program. This program allows the Agency to remain relevant and responsive to client needs that may not be fully met by the core statistical program. In relation to this, the program is able to provide on-demand, high-quality, cost-recovered statistical services including statistical surveys and related services, and non-survey projects.
Statistics Canada is considered a common service organization for optional services under the Treasury Board of Canada (TB) Common Services Policy. An optional service is an activity to supply goods or services that other federal departments and agencies may use when it makes sense to do so. The Agency is authorized to provide optional statistical services to other federal departments and agencies and to offer these same services to Crown corporations or non-federal organizations. This covers developing statistical information through surveys or the use of administrative records, including design, organization, statistical evaluation, interpretation and analysis, and dissemination. The Agency is to provide advice or assistance, or undertake projects, as required. These optional services are to be provided on a contractual cost-recovery basis, and rates charged are to recover but not exceed full cost of the services provided. Further, these optional services must be consistent with the Agency's mandate.
In 2017/2018, the Agency conducted approximately 300 statistical surveys and related services projects, and a large number of custom requests and workshops. Statistics Canada's full cost in providing these cost-recovery activities was approximately $120 million, which was recovered through contractual arrangements with service recipients (i.e., "clients").
The Agency uses the full-cost approach to estimate and negotiate the price of statistical services. Full cost is defined as the sum of all costs, direct and indirect, incurred by the Agency in the supply of a service. The Agency has developed a costing guideline to support the development of costing information used in cost-recovery project estimates. The Statistics Canada Costing Guideline is based on the TB Guidelines on Costing.
Statistics Canada undergoes scrutiny and is challenged by clients in relation to the cost of statistical services. Given this, the appropriateness of the Agency's costing methodology and approach to estimating project costs for statistical services has been questioned periodically.
Statistics Canada continues to strategically position its cost-recovered statistical services to ensure its sustainability by providing high-quality products and services that add value. Operational effectiveness, however, is predicated on having a cost-recovery governance framework that promotes responsive client services and relevant information gathering and analysis. In relation to this, pricing to clients must be based on cost estimates that accurately reflect the Agency's full cost of providing statistical services. To this end, effective protocols, procedures and processes must be in place.
Audit objectives
The objectives of the audit were to provide the Chief Statistician and Statistics Canada's Departmental Audit Committee with reasonable assurance that
- Statistics Canada has established an adequate and effective governance framework that is consistently applied to manage cost-recovery projects and activities
- effective processes and controls to manage cost-recovery projects and activities are in place and consistently applied, in compliance with relevant legislation, policies, directives and guidelines.
Evaluation objective
With assistance from the Evaluation Division, the audit also assessed cost-recovery processes from the client's perspective. The objective of this line of enquiry was to provide the Chief Statistician and Statistics Canada's Performance Measurement and Evaluation Committee with insight on the effectiveness and efficiency of cost-recovery processes from the client's perspective.
Scope
The scope of the audit encompassed the functional direction provided by Corporate Services on cost recovery, and the application of this direction by the Fields during fiscal year 2017/2018.
Approach and methodology
The audit approach included an assessment and analysis of relevant documentation, and interviews with senior management, various managers and staff within the Finance Branch, and divisions in Fields 5, 6, 7 and 8. A sample of cost-recovery projects from nine divisions within Fields 5 and 8 were selected for detailed testing to ensure that projects were managed to achieve expected results and in compliance with Statistics Canada's contractual obligations, and relevant legislation and policies. The sample of projects was selected using a judgmental sampling approach that considered the size and complexity of the project. The sample only included projects within the statistical surveys and related services category (see Appendix C: Sampling methodology).
Through questionnaires and interviews, the Evaluation Division assessed the current levels of client satisfaction with cost-recovery processes. Where clients opted not to use Statistics Canada surveys or statistical services, the Evaluation Division identified the principal reasons behind these decisions.
Authority
The audit was conducted under the authority of the approved Statistics Canada Integrated Risk-Based Audit and Evaluation Plan 2018/2019 to 2022/2023.
Findings, recommendations and management response
The client's perspective
Clients are very positive towards many aspects of the Agency's cost-recovery processes, and most clients intend to continue using Statistics Canada's statistical services. Clients generally choose Statistics Canada as a service provider for data quality and reputation reasons.
Two areas for client satisfaction improvement were identified: (1) clarity and transparency of costs; and (2) access to data prior to release.
Clients responded very positively with respect to many of the Agency's cost-recovery processes, though two areas for improvement were identified.
Statistics Canada conducts a high volume of cost-recovery activities and the public profile of this client-facing program is high. As the Agency moves to be more outward facing and strategically positioned relative to its clients, the client's perspective can yield important insight. To this end, the audit included a client-focused line of enquiry that was led by the Evaluation Division.
The main objective of the client-focused line of enquiry was to provide neutral evidence-based insight on external clients' views of the effectiveness and efficiency of cost-recovery processes. In collaboration with nine divisions within Fields 5 and 8, the Evaluation Division selected a sample of 26 past and ongoing cost-recovery clients for interviews. Clients were eager to participate and express their views, as evidenced by the 92.3% response rate. Interviews were also conducted with three clients who opted not to use Statistics Canada's services.
A key concern of senior management is that existing service recipients may choose to no longer use Statistics Canada's statistical services. One reason often cited is the project cost charged by Statistics Canada to its external cost-recovery clients. One particular example was noted where a significant cost-recovery project is now being conducted by a private sector provider. In this case, the private sector provider's lower price was thought to be a critical factor in the client's choice to change service providers. Senior management expressed concern that cost-recovery clients who chose private sector alternatives were not considering the lower quality of statistical services provided by these private companies.
Overall, existing clients responded very positively (i.e., were satisfied or very satisfied) with respect to many of the Agency's cost-recovery processes (see Appendix D: Client satisfaction), and most clients were not considering other service providers. Notably, 100% of clients were satisfied or very satisfied with the usefulness of the final deliverables and results they received. Other areas with strong client satisfaction included the respect and courtesy of Statistics Canada staff (96%), and the clarity of written and person-to-person communications (91.7%). Interview results further indicated that clients generally chose Statistics Canada as a service provider for non-cost-related considerations, with data quality and reputation being the most often-mentioned reasons. While high cost was cited as a consideration for those choosing not to use Statistics Canada, other factors were also considered (e.g., access to data and timeliness).
While clients were mostly very positive in their responses, two common areas for improvement were nonetheless identified: (1) clarity and transparency of costs; and (2) access to data prior to release.
Clarity and transparency of costs
Only 40% of clients (10 out of 25 who responded) were satisfied or very satisfied with the Agency's clarity and transparency of costs. This represented the lowest-rated question across the entire questionnaire. The audit found no Agency-wide standards concerning when and how costs should be presented to clients. A review of contractual agreements found that a variety of approaches were used to present costs to clients, depending on the specific agreement. Most agreements reviewed did not present costs in a manner that would allow clients to easily understand how the costs for individual components of a project were calculated. For example, some agreements included the total cost estimate for individual project activities (e.g., sample preparation, system development and collection), while others included only lump sum milestone payment amounts without any cost breakdown.
Access to data prior to release
Only 53.3% of clients (8 out of 15 who responded) were satisfied or very satisfied with the level of access to data prior to release. This represented the lowest-rated question for the Project Delivery and Follow-up phase of the questionnaire, and also the question with the highest percentage of negative responses across the entire questionnaire (40% of clients were dissatisfied or very dissatisfied). A review of relevant contractual agreements revealed that the Agency's general terms and conditions do include high-level clauses relating to access to data prior to release. For example, section 20 indicates that an Advance Release Submission has to be established when a client has access to data prior to release. However, many contractual agreements did not otherwise address client access to data prior to release in relation to the project. Generally, clients indicated that better access to data prior to release would provide them with increased information about the responses received and permit them to participate in the data validation process.
Consistent with the Agency's client-service orientation, Statistics Canada can improve the clarity and transparency of its costing methodology. In addition, the Agency can further improve client satisfaction by exploring ways to address concerns regarding access to data prior to release.
Recommendation
It is recommended that the Assistant Chief Statistician, Corporate Services and Chief Financial Officer, in collaboration with all Assistant Chief Statisticians, ensure that
- a standard approach is implemented for communicating costs to clients, and sufficient detail is provided within contractual agreements to enable clients to determine how the total cost of each project is calculated.
It is recommended that the Assistant Chief Statistician, Corporate Services and Chief Financial Officer ensure that
- management explores ways to increase access to data prior to release if possible. In addition, greater clarity should be provided in advance, such as specific terms and conditions, so that clients understand what they have access to, at what stage, and why.
Management response
Management agrees with the recommendations.
Consultations will be conducted with a sample of external cost-recovery clients and sponsoring and supplying divisions within the Agency to establish an appropriate level of detail of costs to provide in letters of agreement. The process for managers to follow when communicating costs to external cost-recovery clients will also be clarified, as will practices for analyzing and giving clients different costing options to determine best value for money. All actions will be conducted with oversight from the Operations Committee.
Deliverables and timeline
The Director General, Finance, will- document and communicate best practices for providing sufficient cost details to external cost-recovery clients and implement a standardized approach for completing letters of agreement by fall 2019
- document and communicate best practices for providing different costing options to external cost-recovery clients by fall 2019.
The Statistics Canada Policy on Official Release makes allowances for cost-recovery clients to have access to anonymized pre-release aggregated data. Inherent to increased pre-release access, however, is the risk of information being inadvertently divulged prior to official release. Additional effort on the part of the Agency is required to increase awareness of flexibility within the policy, while mitigating the risks inherent to increased sharing. All actions will be conducted with oversight from the Operations Committee.
Deliverables and timeline
The Director General, Communications and Dissemination, will- develop a fact sheet, to include with all letters of agreement, to clearly explain the mechanisms already in place to provide access to pre-release data and the measures that must be taken by recipients to fully protect the information until its official release by Statistics Canada by March 2019
- deliver a presentation to subject-matter experts to increase awareness and ensure that options within the Policy on Official Release are well understood and exercised where possible and that information is included in individual letters of agreement clearly indicating what pre-release data external clients will have access to (if any) and when by March 2019
- include a standard clause in the letter of agreement template that refers to the Policy on Official Release by fall 2019.
Governance and risk management
Roles and responsibilities have been clearly defined and communicated. Responsibilities for the delivery and financial management oversight of cost-recovery projects are shared and managed jointly at various levels across the Agency.
The quarterly program financial review process is the key control to validate the quality of project cost estimates. The review process allows Statistics Canada to monitor cost-recovery project costs to budget and know whether a particular project is forecasting a surplus or deficit.
Formal, Agency-wide risk management processes are in place, though there is no Agency-wide mechanism to consolidate, store and analyze information on trends and changes in cost-recovery activities during a given fiscal year to help ensure that Statistics Canada remains relevant in meeting evolving customer needs.
Roles and responsibilities for the delivery and financial management oversight of the Agency's cost-recovery activities have been clearly defined and communicated.
Statistics Canada employs a matrix organizational structure, which interconnects program responsibilities with functional responsibilities. Responsibilities for the delivery of cost-recovery projects are shared and managed jointly by a "sponsoring" division and many "supplying" divisions. The sponsoring division is the subject-matter lead and project manager (e.g., Agriculture Division, Health Statistics Division and Special Surveys Division). Supplying divisions are functional units responsible for providing specialized services that contribute to the delivery of a project (e.g., Statistical Information Systems Division, Household Survey Methods Division, Collection Planning and Research Division). The Finance Branch also plays an important role by defining the Agency's costing methodology and facilitating financial oversight. Further, the Corporate Planning Committee is responsible for providing strategic direction on the operationalization of Agency-wide program and corporate planning activities, and has ultimate financial management oversight responsibility.
The audit team selected a sample of cost-recovery projects for detailed testing. The audit found that all project cost estimates and contractual agreements had been reviewed by the sponsoring division's Financial Officer and approved by the division's Director. Further, for all projects tested, the individuals that signed the contractual agreement had been delegated the authority to do so (initiation authority [Director of sponsoring division] and transaction authority [Revenue Management Team]).
Overall, the audit found that roles and responsibilities for key individuals involved in cost-recovery activities have been clearly defined and communicated. Understanding of roles and responsibilities was confirmed through interviews and evidenced by the results of project testing.
The quarterly program financial review process enables Statistics Canada to monitor variances between a cost-recovery project's actual costs and its estimated budget.
Regular financial reviews should enable management to periodically compare project results achieved against expectations. By having access to reliable financial information and information on project performance, risks and issues, divisional decision makers will be positioned not only to make informed decisions, but also to ensure that appropriate strategies are developed and areas for improvement are identified.
The audit found that monitoring actual costs against a project's budget is the joint responsibility of the sponsoring division, supplying divisions and Finance Branch. It is conducted primarily by way of the quarterly program financial review process, which consists of two relevant steps: supplier review and sponsor review. Financial Officers are assigned to specific divisions and work together to complete the supplier and/or sponsor review steps.
The quarterly program financial review process is therefore the key control to validate the quality of cost estimates. As a result of the quarterly program financial review process, Statistics Canada is in a position to know whether a particular project is forecasting a surplus or deficit. Further, the combination of reporting via the Agency's annual cash/cost analysis exercise, Departmental Results Report and the quarterly program financial review process also helps to ensure that revenues received at the program level do not exceed the Agency's full cost.
The Agency can better support risk management and decision-making by improving processes to capture and report on business and market intelligence.
As previously mentioned in "The client's perspective" beginning on page 9, the risk of existing service recipients choosing to no longer use the Agency's statistical services was highlighted to the audit team by senior management. Through meetings and follow-up with nine sponsoring divisions, the audit identified only four instances (out of approximately 300) of lost clients/projects during 2017/2018. The results of client interviews also suggest that this risk is not materializing in a significant way. Nevertheless, it is important for the Agency to have sufficient business and market intelligence to facilitate the analysis of its strengths and weaknesses to make informed decisions on its cost-recovery activities. Management should identify and evaluate risks that may preclude the successful execution of quality cost-recovery activities, and risk management strategies should be developed and communicated to key stakeholders.
The audit found that formal, Agency-wide risk management processes are in place. Fields prepare individual risk registers, which help to inform the Agency's Corporate Risk Profile (CRP). The 2018/2019 CRP identifies four key risks facing the organization. One key risk within the CRP is the Agency's potential loss of relevance resulting from not fully meeting the needs of its users. In response to this risk, Statistics Canada has launched the modernization initiative, which focuses in part on user-centric service delivery. Field risk registers also include risks that link to the Agency's key risk relating to relevance.
Follow-up with nine sponsoring divisions, however, confirmed that there is no Agency-wide mechanism to consolidate, store and analyze information on trends and changes in cost-recovery activities during a given fiscal year. That is, the Agency relies on the memory of its staff (i.e., anecdotal recollection) to understand where increases or decreases in cost-recovery activities are occurring. This limits the Agency's ability to support the development of plans and strategies to help ensure that the Agency remains relevant in meeting evolving customer needs.
Recommendation
It is recommended that the Assistant Chief Statistician, Corporate Services and Chief Financial Officer ensure that
- the Agency develops processes to provide management with strategic information on its cost-recovery activities in an effort to support decision making.
Management response
Management agrees with the recommendation.
A new Agency-wide requirement will be added to the existing financial management framework to obtain and analyze strategic information regarding changes in cost-recovery activities, such as new contracts, renewal of contracts, lost business and other trends. Strategic information will be captured during program review, and analysis will be included as part of the multi-year planning process. All actions will be conducted with oversight from the Operations Committee.
Deliverables and timeline
The Director General, Finance, will
- develop and use a cost-recovery business intelligence register by fall 2019.
Processes and controls
Statistics Canada has developed Agency-wide standard costing rates that are reviewed regularly and approved by the Corporate Planning Committee on a yearly basis. The standard costing rates fairly reflect the Agency's relevant costs.
Common tools and templates are consistently used to develop project cost estimates and enter into contracts with clients. The Agency works collectively to deliver project results to clients at the agreed-upon price, and material project level surpluses or deficits are rare.
The effectiveness of quarterly program financial reviews could be negatively impacted by some existing time coding practices.
Statistics Canada has developed Agency-wide standard costing rates that fairly reflect the Agency's relevant costs.
The determination of standard costing rates is complex and is based on cost accounting principles, assumptions and management estimates. It is important for Statistics Canada to have developed sound standard costing methods that enable the Agency to ensure that the amounts recovered from external clients fairly represent the Agency's true full cost for rendering its services. Appropriate and consistent standard costing methods and assumptions should be used to establish baseline standard costs in accordance with relevant policies, directives and guidelines.
The audit found that a cost-recovery project's total cost consists of both direct costs and indirect costs. Direct costs include the salaries of employees who will be working on the project and all direct non-salary costs (e.g., travel and translation). Indirect costs are costs incurred to support more than one program or activity (e.g., internal service costs, such as human resources management and financial management). A project's total calculated cost is the amount that Statistics Canada charges and recovers from the client.
The Agency's costing methodology is well documented in the Statistics Canada Costing Guideline. To calculate a cost-recovery project's total cost, Statistics Canada has developed Agency-wide standard costing rates. The Agency's standard costing rates fall into three categories: average salary rates; resource unit rates (RURs); and the Contribution to Indirect Costs (CIC) rate. The CIC rate is used to recover the Agency's indirect costs associated with its cost-recovery activities. For 2017/2018, total indirect costs were calculated to be approximately $17.4 million, excluding associated Employee Benefit Plan costs. In developing the CIC rate, Statistics Canada followed a seven-step approach, as recommended by the TB Guidelines on Costing. All standard costing rates are approved annually by the Corporate Planning Committee.
The audit team reviewed average salary rate, RUR and the CIC calculations for 2017/2018, and found that the methodology used was sound and in line with the Agency's costing guideline. A review of the Agency's annual cash/cost analysis exercise further demonstrated that these standard rates fairly reflect the Agency's relevant costs.
Common tools are consistently used to develop project cost estimates and enter into contracts with clients. The Agency works collectively to deliver project results to clients at the agreed-upon price.
It is important that Statistics Canada have processes in place to ensure that each project cost estimate adequately reflects the level of effort and risk related to the complexity of the project being undertaken to avoid project surplus or deficit situations that could result in a loss of credibility and reputation. Cost estimates should be developed in accordance with the Agency's established costing practices and at the appropriate level of detail for each project. Processes should also be in place to ensure that terms and conditions of contractual agreements are aligned with the Agency's costing practices.
The audit found that, in developing a project cost estimate, a sponsoring division must estimate the level of effort (i.e., number of days per employee group and level) necessary for its own staff and also obtain quotes on the level of effort required from relevant supplying divisions. This information is entered into the Corporate Costing Workbook (CCW), a common tool within which the Agency's standard costing rates are integrated, and the total project cost is calculated. The Agency then prepares a Letter of Agreement (LOA), a formal contract between Statistics Canada and a client.
The audit team selected a sample of cost-recovery projects for detailed testing. For all projects tested, the audit found that costing rates used within the CCWs aligned with the approved Agency-wide standard costing rates. Mathematical accuracy of project cost calculations within the CCWs was also observed. The audit found that LOAs were consistently developed using common templates that included standardized general terms and conditions. Further, the audit observed alignment between a project's total cost estimate, as per the CCW, and amounts identified in the LOA's terms of payment.
With respect to the manner in which individual divisions determine the level of effort necessary for a particular cost-recovery project, methods generally varied among and within divisions, as did supporting documentation. Many sponsoring divisions determine the necessary level of effort based on personal past experience, a previous year's CCW (for recurring and cyclical projects) or level of effort for similar projects. Some supplying divisions have developed standardized tools to assist in developing their level of effort estimates (i.e., quotes), though approaches can vary from project to project and Chief to Chief for others.
All the same, the audit found that scenarios where surplus money is either returned to clients or additional money is requested from clients are quite rare. Though divisional variances were commonly observed within the individual projects tested, the audit only observed material project-level variances for 6 of the 21 projects sampled. That is to say that some division deficits are offset by other division surpluses and, overall, the Agency works collectively to deliver project results to clients at the agreed-upon price.
The effectiveness of the quarterly program financial review process, the key control to validate the quality of cost estimates, could be negatively impacted by some existing time coding practices.
Typically, project cost estimates consist primarily of direct salary costs for both the sponsoring division and relevant supplying divisions. Accurate time coding is therefore essential to assess a project's performance against initial expectations (i.e., to determine and explain variances between actual project costs and budget). In relation to this, each cost-recovery project is assigned a unique program element (PE) code by Finance. Statistics Canada employees in both sponsoring and supplying divisions use the Time Management System to code and record the time they spend working on individual cost-recovery projects (i.e., PE codes). Costs are attributed to both the employee's division and the project.
The audit team conducted interviews with nine sponsoring and supplying divisions that included a discussion of time coding practices within each division. The audit found that four divisions relied on the budgeted amount as the upper limit of actual labour costs to be recorded to a project. That is, staff record the time spent on the project up to the cost-recovery project's budgeted amount, not per the actual effort required to complete the work. For some projects, this practice could lead to an understatement of actual costs. These time coding practices prevent Statistics Canada from obtaining the information required to assess cost-recovery project performance against initial estimates in an accurate and consistent manner.
It is important for employees within sponsoring and supplying divisions to record actual time spent working on individual cost-recovery projects as accurately as possible so that actual effort can be tracked and compared to original estimates. Consistent and accurate time coding would improve the quality of quarterly program financial reviews, and allow the Agency to identify risks, best practices and areas of improvement.
Recommendation
It is recommended that the Assistant Chief Statistician, Corporate Services and Chief Financial Officer, in collaboration with all Assistant Chief Statisticians, ensure that
- consistent Agency-wide cost-recovery project time coding practices that include actual project labour costs are adopted.
Management response
Management agrees with the recommendation.
Enhancements to time coding practices to ensure consistency across the Agency will be explored to effectively report the actual effort required to complete cost-recovery activities. Time coding should represent the actual effort required to complete the work, regardless of the budgeted amount. This practice will be clearly articulated, and awareness will be increased with cost-recovery managers. Analysis will also be performed periodically to identify cost-recovery projects that match expenses to budget. All actions will be conducted with oversight from the Operations Committee.
Deliverables and timeline
The Director General, Finance, will
- increase awareness of time coding practices with cost-recovery managers through COMM-Weekly messages and other communications activities by March 2019
- clarify time coding practices for cost-recovery projects by adding a new Hub page and by including a new section in the Statistics Canada Costing Guideline by March 2019
- implement a new requirement in the program review to analyze and identify projects that match expenses to budget by fall 2019.
Appendices
Appendix A: Audit criteria
Control objectives / Core controls / Criteria | Sub-criteria | Policy instruments/Sources |
---|---|---|
Objective 1: Statistics Canada has established an adequate and effective governance framework that is consistently applied to manage cost-recovery projects and activities. | ||
1.1 Oversight roles and responsibilities are established and clearly communicated with respect to cost-recovery activities. | 1.1.1 Roles and responsibilities related to cost-recovery activities are formally established and understood. 1.1.2 Annual rates, project cost estimates and letters of agreement are reviewed and approved at the appropriate level |
|
1.2 External and internal environments are monitored to obtain information that support decision making regarding the Agency's cost-recovery activities and strategy. | 1.2.1 Regular financial reviews enable management to compare project results achieved against expectations on a periodic basis. 1.2.2 Adequate business and market intelligence is available to facilitate the analysis of Agency strengths and weaknesses and to support decision making. |
|
1.3 Management identifies and evaluates its risks, and risk management strategies are developed and communicated to key stakeholders. | 1.3.1 Management identifies the risks that may preclude the successful execution of quality cost-recovery activities. 1.3.2 Risks related to cost-recovery activities are assessed and mitigation strategies are developed and communicated. |
|
Objective 2: Effective processes and controls to manage cost-recovery projects and activities are in place and consistently applied, in compliance with relevant legislation, policies, directives and guidelines. | ||
2.1 Appropriate and consistent standard costing methods and assumptions are used to establish baseline standard costs in accordance with relevant TB policies, directives and guidelines. | 2.1.1 The methodology used to calculate standard cost rates is documented and follows relevant TB policies and guidelines. 2.1.2 Cost components (i.e., direct and indirect, fixed and variable) used in the calculation of standard cost rates (average salary, RURs and the CIC) are properly identified and classified to ensure project costs fairly reflect the Agency's full costs. 2.1.3 Cost components are not double counted. 2.1.4 Processes in place enable management to ensure that revenues received do not exceed the Agency's full costs. |
|
2.2 Effective processes and controls are in place over the management of cost-recovery activities, including project cost estimates and negotiation of contractual agreements with clients. | 2.2.1 Cost estimates are developed in accordance with Agency costing guidelines at the appropriate level of detail for each project. 2.2.2 Controls in place ensure that project costs and staff time worked on projects and coded accurately to ensure that information required to review project performance against initial estimates is accurate. 2.2.3 Terms and conditions of contractual agreements (LOAs and MOUs) are aligned with the Agency's costing practices. |
|
2.3 Processes and controls in place over the management of cost-recovery activities are in line with the Agency's values, ethics and code of conduct. | 2.3.1 Management has established cost-recovery processes that are in line with Statistics Canada's values and ethics and code of conduct. |
|
Objective 3: Cost-recovery processes are effective and efficient from the client's perspective (assessed by the Evaluation Division). | ||
3.1 Overall, current clients are satisfied or very satisfied with all aspects of the project initiation or planning phase. |
Current clients will be asked to identify their level of satisfaction with
|
|
3.2 Current clients are satisfied or very satisfied with all aspects of the contract or Letter of Agreement phase. |
Current clients will be asked to identify their level of satisfaction with
|
|
3.3 Current clients are satisfied or very satisfied with all aspects of the project delivery and follow-up phase. |
Current clients will be asked to identify their level of satisfaction with
|
|
3.4 Current clients are satisfied or very satisfied with all aspects of the overall assessment of the project. |
Current clients will be asked to identify their level of satisfaction with
|
|
3.5 Clients who have opted to not use Statistics Canada surveys or statistical services identify the principal reasons for their decision. |
Clients who have opted to not use Statistics Canada services will be probed to determine if the principal reason(s) for not doing so were
|
Appendix B: Initialisms
Initialisms | Description |
---|---|
CCW | Corporate Costing Workbook |
CIC | Contribution to indirect costs |
CRP | Corporate risk profile |
LOA | Letter of Agreement |
MAF | Management Accountability Framework |
OCG | Office of the Comptroller General |
RUR | Resource unit rate |
TB | Treasury Board of Canada |
Appendix C: Sampling methodology
Category | Field | Revenue $ | % | # of Projects | Sample size | |
---|---|---|---|---|---|---|
Audit | Evaluation | |||||
Statistical surveys and services | 8 – Social, Health and Labour Statistics | 76,152,440 | 62.4 | 121 | 13 | 18 |
5 – Economic Statistics | 32,884,481 | 27 | 108 | 8 | 5 | |
6 – Analytical Studies, Methodology and Statistical Infrastructure | 7,390,587 | 6 | 65 | Out of scope | ||
3 – Corporate Services | 660,454 | 0.5 | 1 | |||
7 – Census, Operations and Informatics | 212,639 | 0.1 | 2 | |||
Sub-total (SSRS) | 117,300,601 | 96 | 297 | 21 | 23 | |
Custom requests and workshops | All fields | 4,275,983 | 4 | Out of scope | ||
Grand total | 121,576,584 | 100 |
Appendix D: Client satisfaction with cost-recovery processes (select aspects)
Overall, how satisfied or dissatisfied were you with: | Satisfied or very satisfied Positive responses |
Neither satisfied nor dissatisfied Neutral responses |
Dissatisfied or very dissatisfied Negative responses |
Total | |
---|---|---|---|---|---|
The usefulness of the final data or results. | Count | 16 | 0 | 0 | 16 |
% | 100 | 0 | 0 | 100 | |
The respect and courtesy of Statistics Canada staff. | Count | 24 | 1 | 0 | 25 |
% | 96 | 4 | 0 | 100 | |
This project. | Count | 23 | 1 | 0 | 24 |
% | 95.8 | 4.2 | 0 | 100 | |
The clarity of written and person-to-person communications. | Count | 22 | 2 | 0 | 24 |
% | 91.7 | 8.3 | 0 | 100 | |
The time required to complete your project. | Count | 20 | 0 | 2 | 22 |
% | 90.9 | 0 | 9.1 | 100 | |
Clarity and transparency in discussions regarding timeliness. | Count | 20 | 2 | 2 | 24 |
% | 83.3 | 8.3 | 8.3 | 100 | |
Clarity and transparency in discussions regarding the scope of the project. | Count | 18 | 4 | 1 | 23 |
% | 78.3 | 17.4 | 4.3 | 100 | |
Your level of access to data prior to release (if applicable). | Count | 8 | 1 | 6 | 15 |
% | 53.3 | 6.7 | 40 | 100 | |
Clarity and transparency in the cost determination process. | Count | 10 | 7 | 8 | 25 |
% | 40 | 28 | 32 | 100 |