Background

The Treasury Board Secretariat's contracting policy applies to all federal departments and agencies. The objective of this policy is to acquire goods and services in a manner that enhances access, competition and fairness, and results in best value or, if appropriate, the optimal balance of overall benefits to the Crown and the Canadian people.

The policy sets out that "government contracting shall be conducted in a manner that will

• stand the test of public scrutiny in matters of prudence and probity, facilitate access, encourage competition, and reflect fairness in the spending of public funds
• ensure the pre-eminence of operational requirements
• support long-term industrial and regional development and other appropriate national objectives, including Aboriginal economic development
• comply with the government's obligations under the North American Free Trade Agreement, the World Trade Organization Agreement on Government Procurement and the Agreement on Internal Trade."

There is a very broad range of procurement requirements at Statistics Canada for areas such as survey management, transportation services, professional services or consulting, and goods for the Census Program, as well as for information technology (IT) hardware and software.

Statistics Canada is subject to the GCR when contracting for goods and services. In addition, TB contract directives and other appendices and sections govern contracting in the public sector. These regulations and directives consist of both mandatory and optional requirements.

At Statistics Canada, the Materiel and Contracts Services (MACS) section facilitates the procurement process; however, much of the accountability for contract management rests with the cost-centre managers, who are considered the project authority. The mission of MACS is to assist the project authority in contracting for goods and services. Its role is to help the cost-centre managers ensure cost-effective acquisition, use and disposal of materiel assets by applying expert knowledge of related Government of Canada and Statistics Canada policies and procedures.

Audit objectives

The objectives of the audit were to provide reasonable assurance to the Chief Statistician and the DAC that

• the framework that supports procurement and contracting activities is appropriate, comprehensive and effective
• procurement and contracting activities comply with applicable policies, procedures, trade agreements, laws and regulations.

Scope

The scope of the audit included the following elements as they relate to sections 32, 33, 34 and 41 of the FAA: roles, responsibilities, accountabilities, authorities, policies, procedures and monitoring mechanisms. Coverage included the acquisition of goods and services. The audit team reviewed contracts under Statistics Canada's delegation of authority that were issued from April 1, 2014, to September 30, 2015. Acquisitions processed through acquisition cards held by MACS were deemed to be of low materiality and low risk, and were excluded from the scope of this audit.

Approach and methodology

The audit work focused on the examination of documents, interviews with key senior management and personnel, and a review for compliance with relevant policies and guidelines.

The field work included reviewing, assessing and testing the processes and procedures in place with respect to procurement activities. A sample of transactions was selected for testing. The sampling strategy included both a systematic approach and a judgmental approach to ensure adequate coverage of the procurement environment. This environment includes the various procurement vehicles used, the dollar-value thresholds and the various types of professional services acquired.

Authority

The audit was conducted under the authority of the approved Statistics Canada integrated Risk-based Audit and Evaluation Plan, 2014/2015 to 2018/2019.

Framework supporting procurement and contracting activities

Procurement and contracting activities should be supported by a framework that includes clear authorities; guidelines; and a control environment that is appropriate, comprehensive and effective. Key elements of the control environment include the establishment of roles and responsibilities for key stakeholders; clear accountabilities for those with delegated authority; and systems, procedures, tools and management oversight to ensure that consistency and rigour are applied throughout the process.

Authorities and guidelines are clearly defined and ensure rigour is applied throughout the procurement process

Policies and directives for procurement are set by the Treasury Board, and guidelines and procedures for procurement are established at the departmental level. The Delegation of Financial Signing Authorities (DFSA) is the instrument of delegation of authority used across the federal government to manage risks associated with procurement and expenditure initiation. It is also used for formal communication of established authorities. At Statistics Canada, authorities for contract approvals are clearly defined and communicated through the DFSA.

A client guide developed by MACS summarizes the TB contracting policy requirements and covers the contracting process. It also includes a description of standard contracting and acquisition methods for the Government of Canada, as well as an overview of the principles, rules and regulations governing contracting. In addition, the Self-service Hub, which is an evergreen portal to access descriptions of all administrative procedures, provides step-by-step procedures on how to request a contract or engage temporary help services. Policies, directives, guidelines and standards are effectively communicated to all relevant stakeholders and reviewed on a continuous basis through the Hub.

Roles, responsibilities and accountabilities are well defined

The Client Guide developed by MACS clearly and comprehensively defines roles, responsibilities and accountabilities in the following key areas:

• approving requests for contracts
• approving payments
• developing statements of work (SOWs) and evaluation criteria
• obtaining bids
• managing contracts
• closing out files.

The roles, responsibilities and accountabilities of the contract authority and the project authority are clearly defined in the TB contracting policy and in the Statistics Canada Client Guide. These guiding principles are also part of the mandatory training for those with project authority or spending delegation, and for contract authorities and contracting advisors. The responsibilities and accountabilities of the contract authority and the project authority are established, documented and made available to stakeholders.

Project authority

The project authority is the client program that is seeking to procure goods and services for the delivery of its programs. The project authority is responsible for establishing the contract requirements. This includes identifying financial resources, the qualifications of required resources, and the timing of contract requirements; obtaining the authorities for committing funds; and certifying the goods and services received. The SOW (for services) or the requirements specification (for goods) is used to document these requirements and serves as the basis for the supplier evaluation and for managing the contract.

The project authority must follow the process established and referenced in the various links on the Hub. The project authority submits a request for procurement services to the Support Centre for Acquisitions (SCA). Upon receipt of the request, the SCA creates a case file, and the request is sent for expenditure-initiation signature (section 32 of the FAA), according to the DFSA. When the request for procurement is returned, the SCA forwards the complete folder to MACS for processing.

For certain contracts, technical expertise is required to ensure that the goods and services acquired have the precise properties, components and qualifications required to meet the intended operational objectives. Statistics Canada has five areas where technical authorities are used during the procurement process:

• IT hardware and software purchases
• IT professional services
• translation and communications services
• library purchases
• printing and publishing services.

The technical authority has a facilitative role in providing the expertise required for certain types of goods and services. It provides technical assistance to project authorities in drafting SOWs and evaluation criteria for requests for proposals. The mandate of the Information Technology Professional Services Provisioning Unit is outlined in the procedures for procuring IT professional services, available on the Internal Communications Network (ICN) and via the Hub.

Contract authority

The contract authority, who is represented by a contracting advisor in MACS, has the delegated authority to sign contracts under section 41 of the FAA. The contract authority is responsible for ensuring an open, fair and transparent process in the awarding of contracts. Contract amendments must also be authorized by the contract authority.

Regional offices have individuals delegated as the contract authority. These employees report to the Collection and Regional Services Branch. They do not report to MACS. MACS provides procurement and contracting training to regional staff, although it does not assume a direct oversight function for regional procurement activities.

The contract authority's responsibilities include the following:

• developing responsive, creative and flexible procurement strategies
• understanding the requirements of its clients and working with them to achieve their operational objectives
• ensuring that the proposed procurement is within the client's mandate
• reviewing and ranking bidders in collaboration with the project authority
• ensuring that competitive requirements are observed according to regulations and policy
• ensuring that departmental signing authorities are observed
• ensuring that all policies and regulations are adhered to.

Upon receipt of a service request, MACS reviews the requirements established by the project authority and determines the appropriate procurement strategy for each case. If the procurement falls within Statistics Canada's delegation of authority, MACS seeks the necessary approvals for the procurement strategy, including the approval of the CRB, depending on the nature of the contract. Procurement details are then entered into the Common Departmental Financial System (CDFS). MACS is responsible for managing both competitive and non-competitive processes. For procurement that exceeds the agency's delegation of authority, MACS sends the request for processing to PSPC through the CDFS. In such cases, PSPC exercises the role of contract authority, and MACS becomes the technical authority.

Interviews with technical authorities and contract authorities confirmed that there is a clear understanding that the contract authority is responsible for the openness, fairness and transparency of the contracting process. MACS contracting advisors confirmed that their role is to take on the primary challenge function in ensuring that rules are abided by. They also play an advisory role, by ensuring that clients seeking to procure goods and services are aware of their responsibilities as project authority. However, they stated that there is ambiguity in how they should formally address issues, concerns or disputes that may arise, and whether these should be escalated to oversight bodies to ensure that they are satisfactorily resolved. The director general of the Operations Branch, who is responsible for MACS, stated that when disputes arise, contracting advisors, along with their chief, work in collaboration with the clients to resolve matters. They follow all the rules and identify, document and communicate any risks to the agency.

Key controls embedded in the procurement process are appropriate and generally effective

A number of systems and key controls are in place to ensure that procurement transactions are processed within the control framework and in accordance with the TB contracting policy and regulations, PSPC and agency policies, and the FAA.

The Service Request Management (SRM) system serves to communicate with internal clients and to store documentation. The SRM system is used for various operations at Statistics Canada. It is mostly used for IT service requests, but it is also used to manage requests and activities related to procurement and to record all relevant actions. In this way, it leaves an audit trail of approvals and ensures compliance with relevant policies and directives.

Contract authorities at headquarters also use two control tools in carrying out their responsibilities throughout the process: the Procurement Planning and Advance Review (PPAR), and the transaction checklist. These were reviewed for relevance and completeness.

The PPAR is a mandatory planning tool used for all procurement over $25,000, excluding local purchase orders, call-ups and procurement to be processed by PSPC. The PPAR is part of the procurement planning process and is used to formally document approvals before the procurement strategy for acquisition contracts is executed. Testing revealed that 15 out of the 16 files with a PPAR had the required approval signatures.

The checklist is a tracking tool that includes a list of all the required steps of the procurement process. It is used to ensure compliance with policy requirements and to ensure that necessary approvals are obtained. Contracting advisors must sign off on each step once it is completed. MACS requires that a checklist be used for each procurement file, as it is considered a key control in its process.

The audit team reviewed the checklist for completeness and relevance. While the checklist does include a comprehensive and relevant list of steps to be completed, it does not include a step for reviewing the openness and fairness of the technical aspects of the SOW and evaluation criteria. The audit team reviewed the checklists in contract files and found that, in many cases, the files had incomplete checklists or no checklist. When key controls, such as the checklist, are not used in lockstep with the other parts of the process, there is a risk that contracting advisors may not be applying a consistent approach in managing procurement files. This could lead to non-compliance with policies and directives.

MACS has established a quality-assurance regime by reviewing a sample of post-contract files to identify, assess, mitigate and manage risks associated with procurement activities and ensure compliance with the FAA, the TB contracting policy and the GCR.

MACS selects approximately 90 files per year on a sample basis. Files from each contracting advisor are included for a detailed peer review. Necessary corrective measures are identified, and contracting advisors are informed, to improve future performance. It should be noted that the MACS quality-assurance process for post-contract file review does not include any contract files from regional procurement activities. Including regional contracting files in the quality-assurance sample would provide some level of assurance on their compliance with policies and directives.

The Contract Review Board plays a key oversight role; however, its scope could be enhanced to provide a greater level of assurance on procurement activities

The CRB is a recommending body to the assistant chief statistician of the Census, Operations and Communications Field for decisions and activities related to procurement and contracting. The CRB is a standing committee established by the authority of the assistant chief statistician of the Census, Operations and Communications Field.

The CRB has a clearly communicated mandate, terms of reference, roles and responsibilities that are available on the ICN. Its mandate includes elements of risk management, strategic direction and oversight for high-risk contracts. The terms of reference of the CRB describe its responsibilities as follows:

• oversight: review, challenge and pre-approve contracts that are high-risk
• risk management: identify, assess, mitigate and manage risks associated with procurement activities, and ensure compliance with the FAA, the TB contracting policy and the GCR
• strategic direction: review and make recommendations on proposals for changes that emerge from new contracting strategies and audit results.

CRB members review information for each file presented to them and provide their approval or denial via email to MACS. The CRB reviews, challenges and pre-approves the following types of contracts, call-ups (against standing offers), and amendments issued by the agency:

• all contractual requirements for goods and services over $200,000.00 (including applicable taxes) requisitioned directly from Statistics Canada, regardless of whether they are to be the subject of competitive or sole-source contracts, including standing offers and supply arrangements
• all contracts that are to be awarded to former public servants in receipt of a Public Service Superannuation Act pension (as defined in the contracting policy)
• all contracts with a value within 10% of Statistics Canada's delegation of authority, as set out in Appendix C of the TB contracting policy, "Treasury Board Contracts Directive"
• all non-competitive contracts for goods or services when both of the following conditions exist:
  • the value is within 10% of $25,000.00 (including applicable taxes)
  • GCR exception 6(b) is used (the estimated expenditure does not exceed $25,000.00, including goods and services tax or harmonized sales tax)
• all advance contract award notices
• amendments to non-competitively awarded contracts that bring the value over $25,000.00 (including applicable taxes).

A review of the CRB's terms of reference revealed that, in addition to the items identified above, MACS selects "high-risk" contracts for CRB review, challenge and pre-approval. The term "high risk" is not defined. The chief of MACS, responsible for vetting high-risk files for escalation purposes, defines high-risk contracts as those that have a high probability of failure in either the performance of the contract or the execution of the procurement strategy. Within the scope of the audit, a total of 61 procurement case files were sent to the CRB for review, which included 3 identified as high-risk files by the chief.

Interviews with staff and management involved in the procurement process identified other risk elements, which are not in the terms of reference of the CRB, but could pose compliance risks:

• The management and resolution of disputes about technical aspects of procurement processes is a risk element.
• Another risk element is the procurement of goods and services not channelled through the current procurement framework (i.e., justified non-compliance). The Finance Branch monitoring team identifies transactions that require justification for non-compliance, but they are not brought to the attention of the CRB. Without this information, the CRB does not have a full appreciation of the risks related to procurement within the agency, and corrective measures cannot be taken or considered.
• The CRB does not have a formal role in reviewing contractual disputes or challenges initiated by the vendor community or caused by disagreements between the contract authority, the project authority and the technical authority. MACS confirmed that there is no conflict-resolution process in place for procurement and that the CRB is not involved in disputes. Such information would be beneficial to the CRB in its oversight and monitoring function. Currently, such situations are dealt with through meetings and discussions among stakeholders without the involvement of the CRB.

To strengthen the involvement of the CRB in the areas of risk management and strategic direction, industry practice suggests that governing bodies should review the following information:

• the results of the MACS quality-assurance review process, including the review of procurement activities carried out in the regions
• mandatory reports currently produced by MACS for external bodies, which include
  • quarterly reports on all service contracts awarded to former public servants in receipt of a pension under the Public Service Superannuation Act (to the minister)
  • the Annual Procurement Activity Report (to PSPC)
  • the Procurement Strategy for Aboriginal Business (to Indigenous and Northern Affairs Canada)
  • the performance report (to PSPC)
• reports on justification for non-compliance (information about transactions that were processed outside the procurement framework).

The audit team noted that the CRB does not currently review the above sources of information produced by MACS, which could benefit the CRB in fulfilling its mandate. This information would enhance the risk-management role of the CRB and provide it with context on general trends related to procurement activities at Statistics Canada. This could be useful when making recommendations with regard to new contracting strategies.

Oversight conducted by the Finance Branch staff is effective in supporting the procurement process

The Finance Branch staff ensure that procurement and contracting transactions are recorded and managed through the CDFS at three verification points:

1. Verification prior to committing funds (section 32 of the FAA)
2. Pre-payment quality-assurance verification (section 34)
3. Post-payment monitoring verification.

Through these control points, Finance Branch staff ensure that all required documentation is on file, including contract authority sign-off under section 41 of the FAA, and appropriate authorizations and approvals by the project authority.

Within the scope of the audit (April 2014 to September 2015), the expenditure process for procurement was administered manually and involved routing physical (or paper) forms and supporting documentation. It required up to six individual signatures per transaction (excluding the signature of the Receiver General once payment requisition makes it to PSPC).

Clear segregation of duties was in place for expenditure initiation, commitment authority (section 32 of the FAA), transaction authority (section 41), certification authority (section 34) and payment authority (section 33). Controls embedded in the CDFS module are effective in ensuring that requests for contracts and the subsequent contracts are approved only by authorized individuals.

To simplify the expenditure process, the Electronic Request Approval (ERA) system was created and began as a pilot project in June 2015. The system was fully implemented in December 2015, and all sign-offs for sections 32 and 34 of the FAA are now recorded electronically. Since the conduct phase of the audit ended in September 2015, the file review sample for the audit included only a few contracts that were processed through the ERA system. The files that had been processed through the new ERA system were compliant with contracting policies, directives and guidelines.

The Finance Branch monitoring team performs a post-payment review of transactions for compliance with the FAA. The team does not assess the appropriateness of the procurement vehicles used, as this is a MACS responsibility. When issues related to approvals under sections 32 and 34 of the FAA are detected, the project authority is notified by email of the situation and is required to take corrective measures to resolve it. This also applies to transactions that are generated outside the procurement framework (i.e., the contract authority did not sign under section 41). These instances are formally documented by the Finance Branch staff and recorded as requiring justification for non-compliance. The Finance Branch monitoring team reports incidents of non-compliance to MACS and the chief financial officer (CFO). After analysis, the CFO decides whether escalation or any remedial actions are required.

The audit team reviewed the transactions requiring justification for non-compliance. The majority of these items originated from subscription renewals and interdepartmental settlements, while others were for unforeseen circumstances or emergency situations that required immediate attention. Consequently, the CFO is required to make decisions on the eligibility for payment outside the procurement framework.

Instances of non-compliance detected by the Finance Branch staff are not currently subject to review by the CRB. As a result, individuals with delegated authority over procurement are not fully aware of risks to the agency associated with procurement and contracting.

Compliance with applicable policies, procedures, laws and regulations

Procurement and contracting activities should comply with relevant legislative requirements, TB and PSPC policies, the GCR, and Statistics Canada's Directive on the Security of Sensitive Statistical Information.

Procurement practices at Statistics Canada are generally compliant with the Treasury Board contracting policy, directives and the Financial Administration Act

A broad legislative and policy spectrum establishes requirements for contracting activities within the Government of Canada. These include legislative requirements, such as contracting regulations and trade agreements, as well as TB and PSPC policy and procedural requirements.

The Government of Canada is committed to open, fair and transparent procurement and real property transactions. The requirement that all procurement and contracting activities be processed through contracting advisors is the key control to identify, assess and mitigate administrative and operational risks related to procurement and contracting. These risks include excessive sole-source contracts, contract splitting, improper use of contracting methods or vehicles, and risks related to the requirements of the Policy on Government Security.

To test the compliance of procurement activities carried out at Statistics Canada, a sample of 59 procurement files was selected for testing. The sample included 50 files from headquarters (MACS) and 9 files from various regional offices. The sample was selected based on a combination of systematic and risk-based judgmental approaches to ensure sufficient coverage of various types of transactions. Coverage of procurement vehicles and purchasing methods obtained through this sample is as follows:

• for MACS at the head office, coverage included competitive processes (5), competitive standing offers or supply arrangements (8), non-competitive processes not involving standing offers or supply arrangements (21), the Professional Services Online automated purchasing tool (1), Task-Based Informatics Professional Services (4), traditional competitive processes (5), and procurement processed through PSPC (6)
• for regional offices, coverage included competitive processes (5), non-competitive processes (3), and procurement processed through competitive standing offers or supply arrangements (1).

Key findings from the contract file review were as follows:

• Procurement vehicle used
  
  The review of 59 procurement files revealed that, in all cases, the procurement vehicle used was in accordance with contracting policies, directives and guidelines. This included 10 files where mandatory standing offers were applicable.
• Sole-source justification and review for indications of contract splitting
  
  Within the sample selected, 23 files involved a sole-source contract. The audit team found that, in all cases, the sole-source justification was on file and well documented.
  
  Interviews with MACS staff revealed that they do not search the vendor database for historical information that could provide an indication of potential contract splitting. The audit team reviewed the historical information on file for each of these suppliers and found no evidence indicating potential contract splitting.
• Security requirements
  
  Statistics Canada is responsible for protecting the confidentiality of all sensitive statistical information to meet its legal responsibilities and support its reputation as a professional and trustworthy organization. Security requirements set by the project authority should be commensurate with the sensitivity of the information being accessed by contractors, in accordance with Statistics Canada's Directive on the Security of Sensitive Statistical Information.
  
  The review of 59 procurement files revealed that there is a process whereby staff of the security unit at Statistics Canada and at PSPC challenge the security requirements set by the project authority. For 56 of the 59 files reviewed, the required documentation was found in the respective procurement files. For the three files where documentation was absent, the audit team confirmed that the documentation for security verification was available as email confirmations to the contracting advisors. In all cases, the level of security set as a requirement was consistent with the Directive on the Security of Sensitive Statistical Information and appropriate for the sensitivity of the information being accessed.
• Statements of work and evaluation criteria used to assess bidders
  
  The review of 59 procurement files revealed that SOWs and evaluation criteria were adequately established and well documented, and that the rationale was documented and included in each procurement file.
  
  Documentation for the evaluation process was generally comprehensive and complete. With the exception of two files, the evaluation was completed in accordance with the criteria in the request for proposals, and the evaluation form was signed and on file as required. The audit also found that MACS applies rigour in documenting cases involving disputes between the contract authority and a project authority, or between the contract authority and the vendor community. This is a good practice, as it leaves an audit trail for review by external stakeholders.
  
  Of the 59 procurement files reviewed, 2 files involved disputes on the use of evaluation criteria, where a complaint from the supplier community was made through PSPC regarding the use or application of evaluation criteria. After reviewing these two cases and following up with interviews with MACS staff, the audit team confirmed that the cases were resolved, but there is no established protocol in place at Statistics Canada for conflict resolution.
• Contract Review Board approval
  
  Of the 59 procurement files reviewed, 9 cases required CRB approval as per documented procedures. In all nine cases, CRB approval was obtained and formally documented on file.
• Authorization by delegated authorities
  
  or 58 of the 59 files reviewed, the Goods or Services Requisition form (form 1920) and the contracts were authorized by a person with the appropriate delegated authority. In one case, the 1920 form was not on file. This was a transaction of low dollar value processed through the library for a membership renewal. Such transactions are identified in the sample-based post-payment review process of the Finance Branch staff.
• Contract amendments
  
  Of the 59 procurement files reviewed, 14 included contract amendments. The audit team reviewed documentation on file in support of amendments and found that, in all cases, amendments were justified and had been issued prior to the end date of the contract.
• Proactive disclosure of contracts
  
  The TB contracting policy requires the disclosure of contracts with a value over $10,000. It also requires the disclosure of an amendment when its value is over $10,000, or when the amendment modifies the initial value of a contract to an amended contract value that is over $10,000.
  
  A review process is in place to ensure that the requirements for proactive disclosure are met. Prior to sending the list of contracts for posting on the website, the chief of MACS and the director general of the Operations Branch review and approve the list. Information that is posted for proactive disclosure is entered on the website by staff of the Dissemination Division. Once it has been entered, and before it is officially made available to the public, the contract information is reviewed again by MACS to ensure that it is correct and complete.
  
  The audit team verified that all contracts that required proactive disclosure had been posted on the PSPC website. Among the contracts included in the sample, 28 of the 59 contracts had a value over the $10,000 threshold and required disclosure. Results revealed that information for 5 of the 28 contracts was inaccurate on the website. Specifically, the published procurement vehicle was not the vehicle that was actually used. In addition, two contracts were not disclosed on the website as required by policy.
• Evidence of deliverables for section 34 of the Financial Administration Act
  
  For all contracts included in the sample, evidence to certify that goods and services were received, as per section 34 of the FAA, was obtained and reviewed. Evidence was initially obtained from the Finance Branch payment files. In some cases, additional evidence of goods and services received was requested from the project authority, based on deliverables described in the SOW. The audit confirmed that there was sufficient evidence in the files to support the receipt of goods and services, which is required for the certification of payments under section 34 of the FAA.
  
  Statistics Canada's procurement and contracting activities were mostly compliant with relevant legislative requirements, TB and PSPC policies, and the GCR. Procurement and contract clauses were aligned with Statistics Canada's Directive on the Security of Sensitive Statistical Information. The current process to meet proactive disclosure requirements needs to be strengthened to ensure compliance.

Appendix A: Audit criteria

Appendix B: Acronyms and initialisms

Background

Under section 3 of the Statistics Act, Statistics Canada is mandated to "to collect, compile, analyse, abstract and publish statistical information relating to the commercial, industrial, financial, social, economic and general activities and condition of the people" and "to collaborate with departments of government in the collection, compilation and publication of statistical information."

The Canadian Centre for Data Development and Economic Research (CDER) was established to encourage the use of Canadian statistical information / data in research initiatives. Operating entirely on a cost-recovery basis since 2011, the program estimates that it will recover anywhere from $700,000 to $1.1 million annually. The CDER program's main objective is to provide researchers whose projects are approved with secure access to business and economic microdata for analytical research. CDER is a repository containing both business and economic microdata files sufficiently detailed for complex analysis.

The CDER program was first launched to federal government affiliated bodies in December 2011. In October 2012, it was extended to non-federal government institutions. To date, more than 60 projects and 100 researchers have made use of the CDER facilities. The CDER program model is similar to that of the RDC program of Statistics Canada's MAD, which processes confidential social microdata, whereas the CDER program does the same with business / economic microdata.

Located within Statistics Canada's Head Office, CDER falls under the responsibility of the Economic Analysis Division (EAD). It enables Statistics Canada to fill a number of data and capacity gaps, which impose limitations with respect to the development and analysis of business and economic microdata. Specifically, CDER has been designed to enable highly trained economists / researchers to conduct important policy-relevant economic research analysis on topics such as productivity, international trade, investment patterns, and firm dynamics, while assuring the confidentiality and security of the data that is used. CDER has one location only.

Researchers wishing to access CDER data must submit a research proposal and be able to cover all associated project costs. The Analysis Coordination Committee (ACC) evaluates proposals on the basis of their congruence with Statistics Canada's mandate. Furthermore, the proposal review process includes an assessment of the outputs of the research project and of the project's impact on the confidentiality of data providers. Research projects that may breach the confidentiality of data providers are not accepted.

Once a research proposal is approved, only the data required for the approved project are provided to the researchers. The data are first stripped of all identifying information, such as names, contact information, business numbers, detailed geography, and industry of data providers, and then shifted in order to create a synthetic database. Batch mode processing, which includes a vetting process by an assigned economic analyst within EAD, is also used in order to ensure that researchers are unable to identify individual observations in their databases.

In addition to existing memoranda of understanding (MOUs) established with other government departments, all researchers must sign a Microdata Research Contract (MRC), which outlines their responsibilities and accountabilities, as well as the nature of the services rendered by Statistics Canada. These MRCs indicate that researchers are deemed employees under the Statistics Act and must abide by all confidentiality requirements to which Statistics Canada employees are subject, including a security screening process, the Statistics Canada Oath, and the Values and Ethics Code for the Public Service.

Audit Objectives

The objectives of the audit were to provide the Chief Statistician (CS) and the Departmental Audit Committee (DAC) with assurance that the CDER program

• has established an adequate accountability framework in support of maintaining the confidentiality of sensitive statistical information / data
• has established effective control mechanisms to safeguard the confidentiality of data sources and to ensure compliance with applicable Treasury Board Secretariat (TBS) and Statistics Canada policies and guidelines regarding information technology (IT) and physical security, with respect to the delivery of its services.

Scope

The scope of this audit included a detailed examination of the adequacy and effectiveness of the systems and processes established to ensure the protection and confidentiality of sensitive statistical CDER information / data. Specific areas examined included the following:

• the assignment of roles, responsibilities and accountabilities, including those of researchers;
• the research proposal review, assessment and approval processes;
• the establishment of Microdata Research Contracts (MRC) and the procedure followed for obtaining required security clearances for researchers;
• the data masking and vetting processes; and
• the physical and IT controls in place for the CDER program.

The audit also assessed the compliance with TBS and Statistics Canada policies and guidelines relating to the security and confidentiality of data.

Approach and Methodology

The audit work consisted of an examination of relevant documentation, interviews with key senior management and personnel, as well as detailed testing of project files to ensure compliance with relevant policies and guidelines.

The field work also included a review, assessment, and testing of the physical security and IT controls in place to ensure the security and confidentiality of sensitive CDER information / data as part of the CDER facility in Statistics Canada's head office.

In order to test the effectiveness and consistency of key controls, a sample of research projects initiated within the last two years was selected.

The audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada, which include the Institute of Internal Auditors (IIA) International Standards for the Professional Practice of Internal Auditing.

Authority

The audit was conducted under the authority of Statistics Canada's integrated Risk-Based Audit and Evaluation Plan 2014/15-2018/19.

Control environment for the administration of the CDER program

Administration of the CDER program is a combination of assigned responsibilities and procedures to control access to, and protect the confidentiality of, data released to external researchers. Access to the facility should be restricted to researchers with valid security clearances and valid contracts. Researchers should have access only to data which have been approved for use for the specific contract.

Authority

The CDER program operates under the provisions of the Statistics Act, in accordance with all the confidentiality rules and requirements that govern Statistics Canada. The CDER program is accessible only to researchers with approved projects who have been sworn in under the Statistics Act as "deemed" employees.

Roles and responsibilities of key personnel for the CDER program have been defined and communicated, and are understood.

The roles, responsibilities, procedures and practices for the management and handling of the CDER program are defined and documented in the Guideline for CDER memo, dated June 23, 2015, and the Step by Step tasks and tips on good practice for CDER analysts memo, also dated June 23, 2015.

Both documents describe the roles and responsibilities for the EAD analyst assigned to a CDER researcher; CDER administrative staff, IT personnel and/or managers; CDER researchers (deemed employees); IT and subject-matter divisions (other than EAD); and internal and external committees. Procedures and practices for the submission, evaluation and approval of research proposals as well as for security screenings, contract set-up, becoming a deemed employee, system testing, and project conduct, and contract extensions are also covered in detail.

Interview evidence confirmed that the roles, responsibilities, and accountabilities of the key personnel responsible for the CDER program are well understood within the program and by key stakeholders.

Elements of CDER's operational structure and sustainability have not been fully established.

The CDER program is physically located in EAD. The director of EAD, as the CDER program manager, is responsible for its development and operation. Assisting the Director is a full-time economist and four analysts from EAD who have multiple responsibilities, including the CDER program. These responsibilities are in addition to their functional responsibilities as EAD analysts. As a result, reliance is on two full-time staff members to manage the program.

Interviews revealed that the program can sometimes have up to six or more researchers making use of its facilities during peak times. A review of the EAD sector risk profile revealed the following risk: "high turnover of staff, leaving a gap in the skill sets needed in CDER to serve clients."

As the program continues to grow and additional infrastructure becomes necessary to ensure both confidentiality and security of the data the program is sharing, there is an opportunity for the program to assess its strategic direction, overall operational needs and knowledge sharing requirements to ensure ongoing sustainability.

Roles, responsibilities and accountabilities of researchers have been clearly defined and are well understood.

Researchers are deemed employees under the Statistics Act by means of the Microdata Research Contract (MRC) between the program and the researchers. Once an MRC is issued, there is a process to ensure that the security clearance for "reliability" status has been obtained for each researcher by his or her sponsoring government or non-government institution prior to the researcher being given access to CDER data.

The MRC outlines the roles, responsibilities and accountabilities of the researcher and the nature of the research to be conducted. Once an MRC is signed, the researcher in question is expected to participate in a formal CDER orientation session, which covers the regulatory framework governing CDER, CDER guidelines, CDER operations, and the responsibilities of deemed employees in those areas. While these sessions are administered by an EAD analyst and are scheduled only periodically, the CDER program manager will provide an informal orientation to new researchers until they can attend the formal session. The orientation session reinforces the information communicated formally through the MRC, as well as researchers' obligations within the existing policies and guidelines, such as Statistics Canada's Policy on Privacy and Confidentiality and Directive on the Use of Deemed Employees.

Interviews with a sample of five researchers revealed that researchers are aware of their assigned roles, responsibilities and accountabilities, and the audit confirmed that they had followed the orientation session prior to being provided access to the data.

A robust project proposal review and approval process and governance mechanisms have been established but were not always supported by complete documentation.

The process for reviewing, assessing and approving research proposals includes the following four steps described in the Projects in CDER - Review of approval process - June 2015 document. They are (1) an assessment of the feasibility and costs of the project; (2) a peer / professional review to assess the professional merit of the project and the researcher's qualifications; (3) an assessment of the project's alignment with Statistics Canada's mandate; and (4) a formal approval process.

The audit tested a sample of eleven MRCs from a population of sixty-five MRCs issued for the period from July 2013 to July 2015. Five ongoing projects and six closed-out projects were selected on the basis of professional judgment. Of the five ongoing projects, two were proposals submitted by government-affiliated institutions and three were from non-government-affiliated institutions. This sample was used to assess compliance with the established process for the assessment of research project proposals, adherence to the CDER program requirements, and approval by appropriate governance bodies.

Testing revealed overall compliance with the existing process, but an inconsistent level of documentation was maintained to support the conduct of, and the results of, the project proposal review and approval process. Specifically, limited documentation was maintained to support the conduct of the assessment of the proposal's costs and feasibility, as well as the peer and institutional review.

Project renewal and approval processes were consistently applied with appropriate approvals.

Statistics Canada's security requires that an active MRC be in place in order for any researcher to be granted access. Furthermore, researchers as deemed employees must comply with the terms and conditions for access to microdata, as per the Statistics Canada Policy on Microdata Access.

Occasionally, projects are extended and require a contract amendment. In such cases, CDER staff receive an extension request from the researcher and verify that the researcher is still affiliated with his or her sponsor and has a valid security clearance for the entire renewal period.

The process for reviewing and approving project extensions was consistently applied. Of the sample of five ongoing projects selected for testing, three had requested project extensions. In all three cases, although there were gaps between the contract end date and the contract amendment date, documentation was maintained to demonstrate that the contract extension followed the correct process and that appropriate approvals were maintained. The access passes of the researchers were suspended during the gaps in contracts.

The audit also noted that an international researcher (IR) was included in one of the three projects that were tested for extensions. A documentation review revealed that the security clearance information for the IR had not been confirmed by the program for the requested extension period.

Enquiry by the audit confirmed that the individual did in fact hold a valid security clearance for the extended period.

Data Stewardship, Confidentiality vetting and Data linking

Internal protocols and controls for the sound management of data should be in place to ensure the protection and safeguarding of Statistics Canada information over the full lifecycle of the information.

Data Stewardship

Effective processes for compiling and creating synthetic databases have been established, but are not documented and are not formally reviewed

To protect the confidentiality of data, real microdata are never released to researchers; only synthetic data are. An EAD analyst compiles the databases required by the researcher once the project proposal is approved.

Synthetic data are created using an algorithm that was developed by the director of the CDER program. Synthetic data are created by reviewing the real data to identify any potential confidentiality issues and determining which data variables need to be preserved. All identifying information is removed from the databases and any specific characteristics of a given company are shuffled. The extension "_syn.dat" is added to the end of the synthetic data file name. This is the key identifier that the data has been converted. All synthetic data are treated as confidential data and can be accessed only in the CDER program area. This process is a key control used by the CDER program to ensure the safeguarding of data sources and to avoid any potential confidentiality issues.

The audit noted that the process for creating synthetic data has not been formally documented and that only the director and the full-time dedicated employee to the program have access to, and knowledge of, the methodology behind the use of this algorithm. Given the restricted number of individuals who can apply the algorithm, there is a potential risk for the loss of knowledge for this key control should these employees leave the division or the organisation.

The audit also noted that there is no requirement for the formal review of synthetic data files to ensure that not only has the data been sufficiently shifted to protect the confidentiality of the data providers, but also that it has been correctly allocated to the proper researcher user ID, as per their approved MRC.

Confidentiality Vetting

Confidentiality vetting is the process of screening research outputs, syntax or any confidential data-related material to assess the risk of a prohibited disclosure. This is done by analyzing whether obvious identification of individual cases or information about individual cases can be inferred or deduced from the statistical output.

The CDER program is a repository of business and economic microdata files of Statistics Canada that are accessible to researchers with approved projects. Effective and appropriate processes and procedures for confidentiality vetting should be in place and adhered to in order to reduce the risk of unwanted disclosure. Confidentiality vetting should be carefully administered by the program analyst, as per the established protocols, to ensure that confidentiality of data is not compromised.

Practices and procedures to conduct and perform confidentiality vetting are carried out, but they have not been documented. There is currently no requirement for a formal review and sign-off of the vetted data results

Researchers review and analyze their synthetic data, and then, request permission from the CDER program to release their research work outside of Statistics Canada. The research work contains analyses performed on the synthetic data in aggregate form.

Prior to the release of any data, CDER staff complete a confidentiality vetting process. The onus is on the assigned EAD analyst to identify situations in which the confidentiality of data could be compromised during processing. These potential confidentiality concerns are embedded within an automated disclosure control application tool developed within Statistics Canada, called G-Confid. This tool provides the appropriate level of protection for confidential data tables, while minimizing the loss of any synthesized information. Training and guidance documentation provided to analysts is focused solely on how to use the G-Confid application tool. However, to a great extent, subjectivity exists within the vetting process, based on the complexity of the research.

Interviews confirmed that confidentiality vetting of data is carried out; however, testing revealed that no documentation is currently available that outlines the vetting steps to follow, including the assumptions and situations to be considered by the EAD analyst. No vetting evidence was available for the five ongoing projects that were tested and for which evidence was requested by the audit team. As well, the audit determined that evidence of formal sign-offs by the CDER program manager of the vetted data results for the five projects were not available.

The audit was informed that CDER is developing a guidelines document and a checklist to be completed by the analysts when vetting the confidentiality of the data outputs.

Data Linking

A robust process for data linkage is in place and consistently followed

Occasionally, researchers make data linkage requests. In order to do so, a record linkage form must be submitted by the program, which includes extensive details about the proposed linkage, including the security measures in place to protect the confidentiality of the linked data. These submissions are prepared through consultations with the researcher and the responsible Statistics Canada program area. Requests are then reviewed by the director of the program area to determine the potential impact of the data linkage request on the confidentiality of the data sources. The request follows an approval process, which includes Statistics Canada's Information Management Division (IMD), the Assistant Chief Statistician (ACS) who is accountable for the program area, and ultimately the Executive Management Committee (EMB), where the Chief Statistician provides consent. This process, along with the responsibilities of the Statistics Canada program areas involved, is found within the Directive on Record Linkage, which is available through the Statistics Canada's Internal Communication Network (ICN).

Of the five ongoing projects that were tested, one of the projects had a linkage request. The audit team requested the request form for data linkage for the project and noted that it had been approved by the ACS for the division and the CS prior to access being provided to the data.

Overall, the audit noted that a robust process is in place, and is consistently followed to verify that data linkage requests do not create opportunities for data sources to be revealed, and that the proper approval process is followed for requests from the CDER program.

Physical and Information Technology Security

Physical and information technology (IT) controls within the CDER program should be comply with applicable Treasury Board and Statistics Canada policies and guidelines.

Physical access to the Statistics Canada facility is secure, but researchers, are not always actively monitored by the CDER staff while on the premises

Once an MRC is signed, a CDER researcher, as a "deemed employee" of Statistics Canada, is issued a photo identification access card that allows him or her to gain physical access to the CDER program physically located in EAD in the R.H. Coats Building at Statistics Canada.

The audit revealed that although the CDER program operates within the same physical security environment as regular programs, it does not have its own designated space with restricted physical access. Designated researcher workstations are housed in open cubicles, whereby some workstations are grouped together, and others are isolated or intermingled with other EAD staff members on the floor.

Interviews revealed that CDER researchers have to be supervised by their EAD analyst when they access the CDER program facilities between 8:00 a.m. to 6:00 p.m., from Monday to Friday. Monitoring of researcher activities is not covered in the current guidelines. Interviews with EAD analysts and a sample of researchers revealed, that, most analysts finish their work between 4:00 and 6:00 p.m., but do not verify whether their assigned researcher is still on the premises before they leave, and researchers do stay, unsupervised, until 6:00 p.m., from time to time.

As a compensatory control, an automatic computer shutdown at 6:00 p.m. used to occur so that researchers could not access data after the close of CDER's hours of operations. However, since a system migration in fiscal year 2015–2016, this functionality was disabled and was not reinstalled. This provides researchers with the opportunity to stay beyond the operating hours of the program, and to continue with their research work unsupervised.

Researchers are not allowed to use cellular phones, but interviews with CDER staff revealed that this is not being actively monitored or enforced by them.

Effective access, identification and authentication safeguards are in place

The CDER program has an established process in place, where an IT user account for a researcher can be created only when a MRC is approved and becomes active. At the MRC's termination date, the IT user account for the researcher is removed, and can be reinstated only in cases where the MRC is granted an extension by the Director of EAD.

Through the set-up of the IT user account and password, researchers are granted access to their research project folder on the CDER server network. Stored in the research project folder are approved datasets created specifically for the project by their assigned EAD analyst. IT activity on the program's server is randomly spot checked through the use of an 'active directory' system in Microsoft.

To confirm that researchers had access only to approved datasets as stipulated in their MRC, the audit judgementally selected and tested the user ID privileges of five researchers, from a sample population of 65 MRCs issued for the audit period of July 2013 to July 2015. Test results confirmed that researchers could access only their research project folder, and that the datasets stored in their folder were in accordance with their MRC. Attempts to view other research project folders on the server using their IT user account and password were denied.

An audit trail has been established, as part of the batch submit system, to log the activities of the researchers when accessing real microdata. Once data are released and available to researchers, EAD staff members assigned to the CDER program are responsible for monitoring researcher access to the microdata; however, there is currently no formal protocol as to how often this monitoring should take place and how the results should be documented.

IT system safeguards are working as intended

Through testing, the audit revealed that the researchers' workstations do not allow access to the Internet or EAD printers. Only a designated workstation located in an isolated area allows access to the Internet. USB ports are disabled, and the workstations do not have wireless keyboards and/or mice.

An assessment of the IT controls in place for the program has not been done

Interviews revealed that neither an IT security, threat and risk assessment (TRA), prior to the launch of the CDER program, nor periodic inspections of the program, were conducted. The absence of a TRA or ongoing security inspections makes it difficult to assess or mitigate vulnerabilities resulting from inadequate monitoring of researchers. In turn, the sound management and protection of the confidentiality of CDER information is compromised.

Appendix A: Audit Criteria

Appendix B: Acronyms