Statistics Canada

Demography Division
Statistics Canada
demography@statcan.gc.ca

October 2013

Skip to text

Introduction
Section I — The Data
Section II — The Data Tables
Section III — Glossary of Terms
Section IV — Geography

Text begins

Migration Estimates From Tax Records For Census Divisions/Census Metropolitan Areas

Introduction

This report presents migration estimates by census division (CD) and/or by census metropolitan area (CMA).  Five-year comparisons as shown on our printed standard tables enable users to see the pattern of movement by Canadians, as well as immigration and emigration flows to and from Canada. The data tables are updated on a yearly basis.  Migration estimates by CMA are available since 1992-93.

Section I — The Data

Data Source

The migration estimates are derived from a comparison of addresses from individual income tax returns for two consecutive years.  The period of reference extends from April of one year to April of the following year.  A summary of the methodology is provided later in this document.

For the most current data release in October 2013, migration data for 2011 – 2012 were derived by comparing addresses supplied on personal income tax returns filed in the spring of 2011 and 2012. 

Timeliness

Migration estimates are available by census division from 1976‑77 (and by census metropolitan area from 1992-93).  Data on international migration and on internal migration are normally available from 15 to 18 months after the income tax deadline. 

The international migration data exclude net temporary emigrants, returning emigrants and non-permanent residents who did not file an income tax return.

An Outline of the Methodology

The data developed from the taxation records are estimates of migration flows between census divisions or census metropolitan areas by gender and broad age groups (under 18, 18 to 24, 25 to 44, 45 to 64 and 65 and over).  Starting with 2011-2012 migration estimates are produced for 2011 census division boundaries For 2006-07 through 2010-11 migration, estimates are produced for 2006 census division boundaries. For 2001-02 through 2005-06 migration, estimates are produced for 2001 census division boundaries. For 1996-97 through to 2000-01, estimates are produced for 1996 census division boundaries. From 1992-93 through to 1995-96, the data were produced for 1991 census division boundaries. For the 1986‑87 through 1991-92 periods, the census division boundaries corresponded to those defined in the 1986 Census. For the years 1981-82 to 1985-86, the boundaries used are those defined in the 1981 Census and in the previous years the 1976 census boundaries are used.

Migration flows for census metropolitan areas are available since 1992-93, and the boundaries of the 26 CMAs are based on 1991 Census definitions for the period 1992-93 to 1995-96.  CMA boundaries based on the 1996 Standard Geographic Coding (SGC) were used in the creation of 1996-97 to 2000-01 estimates.  Beginning with 2001-02, CMA boundaries are based on 2001 SGC.  The CMAs of Kingston (Ontario) and Abbotsford-Mission (British Columbia) were introduced with this new SGC system. Therefore, the 2001-02 to 2005-06, migration data are available for a total of 28 CMAs. For the 2006-07 to 2010-11 period, CMA boundaries are based on 2006 SGC. The CMAs of Moncton (N.B.), Barrie (Ont.), Brandford (Ont.), Guelph (Ont.), Peterborough (Ont.), and Kelowna (B.C.) were introduced with the 2006 SGC system. Therefore, migration data for that period are available for a total of 34 CMAs.  For the 2011-12 estimates, CMA boundaries are based on 2011 SGC.  No new CMA has been introduced with the 2011 SGC therefore migration data are available for a total of 34 CMAs.

The development of these data involves four main steps:

1. Geocoding of tax records;
2. Estimation of non‑filing dependents of taxfilers, by age group and gender;
3. Identification of the number, age group and gender of migrant taxfilers; and
4. Adjustment for the population not covered by the Canada Revenue Agency Taxation system.

Step 1 ─ Geocoding

The geographic coding of census divisions and census metropolitan areas on the tax records is done primarily on the basis of the Postal Code, which is part of the mailing address.  In some cases, other pieces of information were used in place of a missing postal code.  Since the 1989 tax files, over 99% of the records could be assigned a census division code.

Step 2 ─ Estimation of Dependents

Since the source file has no direct information on the number and characteristics of non-filing dependents, this information must be imputed.  Up to the 1987-88 period, this was based on the relationship between the dollar value of the total personal exemptions claimed and the number of dependents.  A reference table was established relating an estimate of the average number of dependents by age group and gender to filers in a given age‑gender‑marital status‑total personal exemption class.  This table was produced each year using a sample file of taxfilers containing information on the exact number of dependents and their relationship to the filer in addition to the characteristics of the filer.  Other demographic information such as gender ratios and the age distributions of husbands and wives were also used to distribute dependents by age and gender.

The current system uses the estimation of taxfilers' dependents from the T1 family file (T1FF).  The family system creates families by linking all filing family members together and estimates non-filing members from information on the taxfilers' returns,^Note1 based on such information as deductions/tax credits for dependents.  For example, the family system imputes a non-filing spouse wherever a filer has declared him/herself married but was not linked with a filing spouse.^Note2

Step 3 ─ Migrant Taxfilers and Dependents

The main source file used contains the basic demographic and geographic information on each taxfiler (and dependent) and covers approximately 95% of the total population.  The migrant taxfilers are identified by comparing current and previous census divisions or census metropolitan areas of residence.

Taxfilers' non-filing dependents are assumed to have the same migration behaviour as that of the filer to whom they are assigned.

Step 4 ─ Coverage Correction

The final step in the estimation process is an adjustment for coverage, done by age and gender at the census division/census metropolitan area level.  Population estimates by CD/CMA are used to create coverage ratios.  For migration estimates up to 2000-01, provincial adjustment ratios were used in place of the CD/CMA ratio in the few cases where coverage was abnormally high or low,  Beginning with 2001-02 migration data, high and low coverage were identified with a new methodology and a Canadian adjustment ratio was used in place of the CD/CMA ratio. Starting with 2006-2007 migration data, adjustment ratios use the CD/CMA ratio.

The adjustment ratios are applied to the counts of out‑migrants derived in Step 3 to obtain an estimate of total migration.  The basic assumption is that the population not covered by the taxation system has the same migration rate as that covered by it.

The estimates of international migration are prorated to agree with provincial estimates provided by the Demography Division of Statistics Canada.

Data Quality

Based on a detailed evaluation of the estimates for the intercensal period of 1986-91, a number of observations can be made regarding migration estimates for Census divisions:

(a) Overall, the estimates of migration are of good quality.  It is, however, difficult to make exact comparisons to other annual estimates of migration flows at the census division level.  The estimates of net migration have been used to produce population estimates and these have been compared to the 1991 Census counts.  The average absolute difference for 1991 was 2.3%.  In 12 of 182 cases (6%)^Note3 the deviation exceeded 5% and in 3 cases, the deviation exceeded 10% (this does not include Quebec census divisions).  These deviations are smaller than those obtained from other estimation methodologies and indirectly indicate the quality of the net migration data.  It has not been possible to do much evaluation of the flow data.

(B)   In addition to the estimates of migration based on tax records, Demography Division of Statistics Canada also produces estimates of interprovincial migration which are based on Child Tax Benefit records.  The concepts underlying these estimates differ from the concepts used in the tax-based estimates.  More specifically, the Child Tax Benefit data estimates monthly moves while the tax data tracks annual moves.

No comparable study has yet been done to examine the CMA coverage.

Availability of Data

Migration estimates are available by census division from 1976‑77, and by census metropolitan area from 1992-93.

For the 1976‑81 period, no preliminary migration flows between census divisions were calculated.  Adjustments were not made at the international level although evaluations indicated the estimates of international migration were too low.

For the period from 1981‑82 to 1984‑85, migration estimates from tax records were produced twice a year, the first time using a preliminary tax file from the Canada Revenue Agency (available with a 6‑9 month time delay) and the second with a more complete tax file (available with a 12‑15 month time delay).  Because the differences between the two sets of estimates were not large, beginning with the 1985‑86 estimates, only one series of estimates was produced.  The final file has been used since 1985-86.

Beginning with the 1981‑82 estimates, the data on immigration and emigration have been prorated to make them consistent with the most currently available estimates produced at the provincial level by Demography Division of Statistics Canada.

Section II — The Data Tables

Number of Tables

There are four standard data tables. The tables provide a five-year comparison of migration:

Table A:  By Province of Origin/Destination
Table B:  By Age Group
Table C:  By Type of Migration and Gender
Table D:  Flows by Census division of Origin/Destination, or by CMA/non-CMA of Origin/Destination

Note: A five-year comparison is not always possible for census divisions due to boundary changes over time.

Data Table Contents

Table A ─ By Census Division or Census Metropolitan Area of Origin and of Destination
Each page of this table highlights flows: in, out and net flows for a specific province, a specific census division (CD) or any one of the 34 census metropolitan areas (CMAs), including non-CMA areas for each province.

Provincial totals provided include intraprovincial migration.

Tables B and C ─ Age Group, Type of Migration, and Gender
Both Tables B and C list in, out and net migration for the highlighted CD or CMA for a five-year period.  Table B shows migration by age group while Table C shows migration by type (interprovincial, intraprovincial and international) and by gender.

Provincial totals provided include intraprovincial migration.

Table D ─ Flows by Census Division/Census Metropolitan Area of Origin/Destination
Table D gives details of the flows for a particular CD or CMA.  A list is given of the CDs or CMAs with which the selected CD or CMA exchanged any people.  The flows are ranked in the table by net migration.  The flows for the past five years are shown on the printed tables, though the ranking is according to the most recent period.

Section III — Glossary of Terms

Age
Is calculated as of December 31 of the reference year (i.e., tax year minus year of birth).

Census division (CD)
Group of neighbouring municipalities joined together for the purposes of regional planning and managing common services (such as police or ambulance services). These groupings are established under laws in effect in certain provinces of Canada. For example, a census division might correspond to a county, une municipalité régionale de comté or a regional district. In other provinces and the territories where laws do not provide for such areas, Statistics Canada defines equivalent areas for statistical reporting purposes in cooperation with these provinces and territories.

Census metropolitan area (CMA)
Area consisting of one or more neighbouring municipalities situated around a core. A census metropolitan area must have a total population of at least 100,000 of which 50,000 or more live in the core.

Dependent
For the purpose of these databanks, dependents are the non-filing members of a family.  We do not attempt to measure dependency in any way, but are able to identify certain non-filing family members, and include these in the total counts of people in a given area.

Emigration
Movement from an area in Canada to another country.

Gross migration flow
Sum of the number of migrants between two geographic areas.  It is obtained by adding the number of in‑migrants to the number of out‑migrants.

Index
Is a comparison of the variable for the given area with either the province (province = 100) or with Canada (Canada = 100).

Immigration
Movement to an area in Canada from another country.

In‑migration
Movement to a census division or census metropolitan area from elsewhere inside or outside Canada.

Internal migration
Movement between two census divisions or census metropolitan areas within Canada.  Internal migration is divided in two categories: interprovincial and intraprovincial migration.

International migration
Movement between an area in Canada and another country.  International migration is divided in two categories: immigration and emigration.

Interprovincial migration
Movement between census divisions or census metropolitan areas located in two different provinces.  The province of departure is the “province of origin” and the province of arrival is the “province of destination”.

Intraprovincial migration
Movement between two census divisions or census metropolitan areas located within the same province.  The CD/CMA of departure is the CD/CMA of “origin” and the CD/CMA of arrival is the CD/CMA of “destination”.

Median
Is the middle number in a group of numbers.

Migration
Movement between two geographic areas during the period covered by the estimates. Within Canada, the geographic area of reference is the census division or the census metropolitan area.  Other countries are considered as one geographic area.

Net migration
Difference between the number of in‑migrants and the number of out‑migrants.

Out‑migration
Movement out of a census division or census metropolitan area to elsewhere inside or outside Canada.

Taxfiler
Most taxfilers are people who filed a tax return for the reference year and were alive at the end of the year.  Starting with the 1993 tax year, those taxfilers who died within the tax year and who had a non-filing spouse had their income and their filing status attributed to the surviving spouse.

Section IV — Geography

The data are available for census divisions and census metropolitan areas.  The mailing address at the time of filing is the basis for the geographic information in the tables.

The following table shows the coded designators for each level of geography, as well as a brief description of each. 

Table 1
Table summary
This table displays the results of table 1 . The information is grouped by level of geography (appearing as row headers), name and description (appearing as column headers).

Level of Geography	Name	Description
41	Census Metropolitan Area	There are 34 CMAs in the 2011 databanks: 001, St. John's, Newfoundland and Labrador 205, Halifax, Nova Scotia 305, Moncton, New Brunswick 310, Saint John, New Brunswick 408, Saguenay, Québec 421, Québec, Québec 433, Sherbrooke, Québec 442, Trois-Rivières, Québec 462, Montréal, Québec 505, Ottawa-Gatineau (Québec part) 505, Ottawa-Gatineau (Ontario part) 521, Kingston, Ontario 529, Peterborough, Ontario 532, Oshawa, Ontario 535, Toronto, Ontario 537, Hamilton, Ontario 539, St-Catharines-Niagara, Ontario 541, Kitchener-Cambridge-Waterloo, Ontario 543, Brantford, Ontario 550, Guelph, Ontario 555, London, Ontario 559, Windsor, Ontario 568, Barrie, Ontario 580, Greater Sudbury, Ontario 595, Thunder Bay, Ontario 602, Winnipeg, Manitoba 705, Regina, Saskatchewan 725, Saskatoon, Saskatchewan 825, Calgary, Alberta 835, Edmonton, Alberta 915, Kelowna, British Columbia 932, Abbotsford-Mission, British Columbia 933, Vancouver, British Columbia 935, Victoria, British Columbia
21	Census Division	The 2011 databanks contain 293 Census Divisions.

Data in many forms

Statistics Canada disseminates data in a variety of forms.  In addition to publications, both standard and special tabulations are offered.  Data are available on the Internet, compact disk, diskette, computer printouts, microfiche and microfilm and magnetic tape.  Maps and other geographic reference materials are available for some types of data.  Direct online access to aggregated information is possible through CANSIM, Statistics Canada's machine-readable database and retrieval system.

How to obtain more information

Inquiries about these data and related statistics or services should be directed to:

Client Services
Demography Division
Statistics Canada
Main Building, 1st Floor
Ottawa, Ontario K1A 0T6
Telephone; (613) 951-2320
Fax: (613) 951-2307
demography@statcan.gc.ca

Advisory Services provides a wide range of services: identification of your needs, establishing sources or availability of data, consolidation and integration of data coming from different sources and development of profiles, analysis of highlights or tendencies and, finally, training on products, services, Statistics Canada concepts and also the use of statistical data.

National enquiries line: 1-800-263-1136
National telecommunications device for the hearing impaired: 1-800-363-7629
Order-only line (Canada and the United States): 1-800-267-6677
National Toll-free Fax line: 1-877-287-4369

You can also visit us on the web:  http://www.statcan.gc.ca.

Standards of service to the public

Statistics Canada is committed to serving its clients in a prompt, reliable and courteous manner and in the official language of their choice. To this end, the Agency has developed standards of service that its employees observe in serving its clients. To obtain a copy of these service standards, please contact Statistics Canada toll free at 1 800 263-1136. The service standards are also published on www.statcan.gc.ca under About Statistics Canada > Providing services to Canadians.

Notes

1. See report "Description of the Methodology Used to Create Migration Data from Tax Records" updated by Judy Reid, Small Area and Administrative Data Division: February 1998.

2. See Lucaciu, Daniela and Harris, Shelley, "Overview of T1FF Processing", SAADD: 1999.

3. Montgomery, March 1993: p 15

Audit Report

April 22, 2013
Project Number: 80590-71

• Executive summary
• Introduction
  • Background
  • Audit objectives
  • Scope
  • Approach and methodology
  • Authority
• Findings, Recommendations and Management Responses
  • Management Control Framework
  • Stewardship
• Appendices
  • Appendix A: 2011 Census Field Structure
  • Appendix B: Pay Process per Form 49-E Finance Office Manual
  • Appendix C: Audit Criteria
  • Appendix D: Acronyms

Executive Summary

The Statistics Act requires Statistics Canada to perform a Census of Population (Census) every five years. The Census is the largest, the most complex and publicly visible program, which obtains information from the entire population of Canada. In June 2010, a decision was taken by the government to maintain the 2011 Census of Population as a basic short-form census. All remaining questions that were previously asked in the long form census were included in a new voluntary survey, the National Household Survey (NHS).

Collection operations for the Census/NHS program operated nation-wide, with 3 regional offices, 5 Regional Census Centres (RCC) and numerous Local Census Offices (LCO) located across the country. Non-Response Follow-Up (NRFU) activities were carried out over a period of 11 to 15 weeks. There are distinct and inherent challenges to managing NRFU activities of approximately 30,000 enumerators operating in every part of the country, while striving to achieve the desired response rate for both the Census and NHS. In order to meet Census program objectives, Census management had the task of striking an appropriate balance between applying vigilance and due care to cost controls while managing the complexities of collection operations, in order to obtain a satisfactory response rate within an inflexible timeframe.

The practice of timekeeping for the enumerators was new in 2011. In the past, enumerators were paid based on piece rate. In order to improve enumerator recruitment and retention for the 2011 Census, the approach was modified to pay enumerators on an hourly rate. Timekeeping became an important element and was integrated into the Field Management System (FMS). Field staff used the FMS to report work performed daily, and to enter their pay and expense claims. Total costs managed through FMS, amounted to approximately $128 million.

This audit took place after Census collection operations had wound down and focussed on key controls and processes in place to ensure the reasonableness of enumerator fees. It was meant to add value by assessing practices in place during the 2011 Census, verifying if identified risks from a management led review had materialized, and identifying potential for improvements as management takes on the planning phase for the 2016 Census.

This report is intended to provide the Chief Statistician and senior management of Statistics Canada with relevant information that supports engagement observations and recommendations. As such, the focus of this report lends itself to the areas which may need management's attention. Areas where performance was deemed satisfactory have been summarized and acknowledged as part of the overall conclusion.

Key Findings

Management has identified a number of risks that may preclude the achievement of its objectives and has assessed the existing controls in place. However, financial risks resulting from significant changes in the Census pay structure were considered but not included as part of the risk management framework.

The change to enumerator pay based on an hourly rate, rather than on piece rate, added complexity to financial planning of collection operations due to a lack of historical data. Regions were instructed to carry out field operations and report on program spending, with the understanding that additional funds would be allocated if required to complete both Census and NHS collection operations based on active monitoring of response rates. The absence of historical data, combined with the changes related to the new NHS, made it challenging for Census management to set informed financial targets for Census 2011. This vulnerability elevates the risk that regional management may have a diminished incentive to control NRFU costs for collections activities.

FMS and Management Information Systems (MIS) provided timely management information for case management and pay/expenses incurred by enumerators. The quality of information contained in these reports can be enhanced by identifying and explaining financial variances so that management can monitor actual performance against planned results and adjust its course as needed.

Training and directives with regard to cost control and the use of MIS reports identified roles and responsibilities but would have benefitted from clearer direction or guidance on the approach to carrying out oversight duties.

Planning and monitoring activities were mainly focused on meeting collection targets. Crew Leaders (CL) and Field Operation Supervisors (FOS) reviewed claims and productivity reports for reasonableness. This key control relied upon by Local Census Office Managers (LCOM) for approving pay and expense claims under Section 34 of the FAA, provided low assurance that costs were reasonable.

Results of audit tests performed demonstrated adherence to the Census Pay Process. All employees examined in the sample were deemed legitimate; all travel claims examined were pre-approved and signed by individuals with the proper authority; and payments were deposited in the bank accounts of the intended enumerators. Enumerator pay and expense claims were batched and approved by individuals holding authority under Sections 34 and 33 of the FAA. However, in most cases, there was no evidence of any analysis or enquiries regarding accuracy and reasonableness of batches, as required for Section 34 sign-off. Consequently, individuals approving payments under Section 33 did not always have sufficient evidence that procedures required under Section 34 had been performed.

Overall Conclusion

The Census is the largest, most complex and publicly visible program at Statistics Canada. It is a collaborative initiative involving both the national program and regional operations. Collection activities are carried out over a very short period of time. The Field Management System (FMS) was developed mainly to address communication problems encountered in 2006 Census. A significant change occurred for the 2011 Census; enumerator pay was based on an hourly rate, rather than on piece rate and timekeeping became an important element and was integrated to FMS.

While Statistics Canada has an adequately designed control framework in place surrounding expenses processed through FMS, opportunities exist to strengthen controls over NRFU costs.

Audit procedures did not reveal any instances of fraudulent activity, malfeasance, or misappropriation of funds.

Key controls relied upon by Local Census Office Managers (LCOM) for approving pay and expense claims under Section 34 of the FAA, provided low assurance that costs were reasonable.

Conformance with Professional Standards

The conduct of this engagement conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the Quality Assurance and Improvement Program.

Patrice Prud'homme
Chief Audit Executive

Introduction

Background

The Statistics Act requires Statistics Canada to perform a Census of Population (Census) every five years. The Census is the largest, the most complex and publicly visible program, which obtains information from the entire population of Canada. It is different from other surveys, in that every household in Canada is legally required to complete the Census form; therefore, it attains higher level of coverage and accuracy than voluntary surveys. In June 2010, a decision was taken by the government to maintain the 2011 Census of Population as a basic short-form census. All remaining questions that were previously asked in the long form census were included in a new voluntary survey, the National Household Survey.

Changes to the Census questionnaire and the introduction of the National Household Survey also introduced changes to the sample decision that resulted in a significant increase in the number of dwellings receiving the Census and the National Household Survey questionnaires. The Census questionnaire was sent to 100% of Canadian dwellings with the National Household Survey distributed to 33% of Canadian dwellings.

In addition to initial funding received for the 2011 Census of Population, Statistics Canada received additional funding that was intended to be used for three purposes:

• Implement changes to the questionnaires and processing systems;
• Increased costs associated with the printing and mailing of additional questionnaires (both Census and the NHS)
• Increased follow up, if necessary, to ensure a 98% response rate on the Census

Statistics Canada hired approximately 30,000 enumerators under the Statistics Act to conduct field collection operations for the 2011 Census and NHS.

The timeframe in which Non-Response Follow-Up (NRFU) activities were carried out was very short. There are distinct and inherent challenges to managing the NRFU activities of approximately 30,000 enumerators operating in every part of the country. In order to meet Census program objectives, Census management had the task of striking an appropriate balance between applying vigilance and due care to cost controls while managing the complexities of collection operations, in order to obtain a satisfactory response rate within an inflexible timeframe.

The timelines for Census and NHS were as follows:

• Early Enumeration: from February 1, 2011 to May 20, 2011 (15 weeks)
• Census NRFU: from May 20, 2011 to August 5, 2011 (11 weeks)
• NHS NRFU: from June 8, 2011 to Aug 24, 2011 (11 weeks)

Collection operations for the Census/NHS program operated nation-wide, with 3 regional offices, 5 Regional Census Centres (RCC) and numerous Local Census Offices (LCO) located throughout the country.

One of the most important innovations introduced for the management of the 2011 Census was the Field Management System (FMS). This system was developed mainly to address the communication problems encountered in 2006, but also to include other features to support collection work in the field.

The practice of timekeeping for the enumerators was new in 2011. In the past, enumerators were paid based on piece rate. In order to improve enumerator recruitment and retention for the 2011 Census, the approach was modified to pay enumerators on an hourly rate. Timekeeping became an important element and was integrated to FMS. Field staff used FMS to report their productivity on a daily basis, and to enter their hours worked and expense claims.

Total costs managed through FMS, which consists of enumerators' salary fees, local KMs claimed and other costs, amounted to approximately $128 million.

Table 1: Total costs managed through the Field Management System (FMS)

Region	Claims	Salary ($)	Local travel ($)	Meals and expenses ($)	Total ($)
*Source: FOP (2011 Census - Field Operations Project)
East	681,016	37,730,179	5,291,290	123,028	43,144,496
Central	696,349	35,777,213	3,968,622	101,008	39,846,843
West	705,949	39,199,572	5,672,516	331,447	45,203,535
TOTAL	2,083,314	112,706,963	14,932,427	555,483	128,194,874

In November 2011 and February 2012, two management-led reviews were performed on the Census Pay Systems: Design and Implementation; and Operating Effectiveness. The reviews covered mostly IT controls imbedded within the FMS system as part of the overall assessment of internal controls for Census Pay. Results of these reviews revealed a number of significant weaknesses related to FMS; however no testing had been undertaken to verify whether existing compensatory controls were effective in mitigating these risks or if risks identified for 2011 had materialized. Management plans to address these concerns as part of the planning phase of the 2016 Census.

Aside from IT controls imbedded within the FMS, other key controls were in place to ensure the reasonableness of enumerator fees, such as reviewing enumerators' performance prior to approving hours and KMs claimed and closely monitoring progress. Monitoring results and promptly addressing performance issues were also key controls relied upon to ensuring program objectives were met. Throughout the Census collection period, the FMS also provided a wide range of reports and information, updated daily, and accessible to supervisors and program management for timely decision making.

The audit took place after Census collection operations had wound down. It was meant to add value by assessing practices in place during the 2011 Census, verify if identified risks had materialized, and identifying potential for improvements as management takes on the planning phase for the 2016 Census.

Audit Objectives

The objectives of this audit were to provide assurance to the Chief Statistician and Statistics Canada's Departmental Audit Committee on:

• The adequacy of the control framework in place surrounding financial planning, monitoring and reporting of expenses processed through FMS; and
• The effectiveness of key controls embedded within the pay process surrounding payments to employees hired under the Statistics Act during the 2011 Census/NHS, including those managed through the Field Management System (FMS).

Scope

The audit focussed on mechanisms in place to ensure the reasonableness of enumerator fees. This included compliance to procedures described in Form 49-E Finance Office Manual, as well as the quality of financial planning, monitoring and reporting practices critical to sound program financial management and reports produced for high-level program decision making.

Two regions were selected for the conduct of this audit: the Eastern and Western regions. These two regions offered the most national coverage for the Census collection process. Two site-visits were conducted during the execution of the audit. Regional headquarters are located in Montreal and Edmonton, and were the focus of this audit.

Approach and Methodology

This audit was conducted following the Standards for the Professional Practice of Internal Auditing as per the Institute of Internal Auditors (IIA) and in accordance with the TBS Policy on Internal Audit. The audit approach was inspired by the Government of Canada Management Accountability Framework (MAF) and the Treasury Board Core Management Control guidelines issued by the Office of the Comptroller General.

The audit work consisted of an examination of documents, interviews with key Senior Management and personnel of Statistics Canada, including the Collection and Regional Services Branch (CRSB), as well as former employees hired under the Statistics Canada Act, a review of the processes and procedures outlined in policies, various guidelines and regional manuals, and substantive testing based on a judgemental sample of transactions.

Authority

The audit was conducted under the authority of Statistics Canada Integrated Risk-Based Audit and Evaluation Plan 2012/13-2014/15 recommended by the Departmental Audit Committee, April 2012 and subsequently approved by the Chief Statistician.

Findings, Recommendations and Management Responses

Objective No. 1: Adequacy of the control framework in place surrounding financial planning, monitoring and reporting of expenses processed through FMS.

Management Control Framework

Management has identified a number of risks that may preclude the achievement of its objectives and has assessed the existing controls in place. However, financial risks resulting from significant changes in the Census pay structure were considered but not included as part of the risk management framework.

The absence of historical data related to the way enumerators were paid, combined with the changes related to the new NHS, made it challenging for Census management to set informed financial targets for Census 2011. This vulnerability elevates the risk that regional management may have a diminished incentive to control NRFU costs for collections activities.

FMS and MIS provided timely management information for case management and pay/expenses incurred by enumerators. The quality of information contained in these reports can be enhanced by identifying and explaining financial variances so that management can monitor actual performance against planned results and adjust its course as needed.

A sound management control framework should be in place to effectively manage Census operations and related risks. Review and monitoring of enumerator performance, deemed a key control, should be conducted on a daily basis and poor performers should be dealt with in a timely fashion. Ongoing reviews, analysis, and reports of financial results versus established budgets should occur to ensure performances of the regional operations are managed appropriately, and communicated to relevant stakeholders for program decision-making.

Risk Management

Roles, responsibilities and accountabilities for risk management were clearly defined at the Census Project Team (CPT) management level. Risks identified covered most applicable types of risks and considered both internal and external sources of risks. Risk events were also identified at the regional level, and captured in the Risk Management System (RMS) by FOP management at NHQ.

The Risk Management System (RMS), specifically created for the Census through a Census Master Risk Register (CMRR), served as a formal risk management system for the 2011 Census. Identified risks and associated mitigation strategies were formally captured in the RMS. Each risk was categorized according to its probability and impact, and a risk register was developed for all high impact risks faced by the program. Specific options and mitigation strategies were developed, discussed, approved and formally documented in advance by the Census project team.

For each identified risk, the following information was documented: risk owner, date risk was identified, category (internal or external), probability, impact, affected activities element and projects, escalated to which activity element (responsible manager), timeframe, detailed description of the risk, impact assessment, mitigation plan and file attachment for contingency plans. A closing response was also provided at the end of the Census.

The audit revealed that financial risks resulting from significant changes in the Census pay structure were not included in the RMS. However, these risks were identified by FOP and regional operations staff, and were documented as part of planning documents and analysis. Examples of risks documented by the program or mentioned by regional staff during interviews included: the inability to effectively manage costs under hourly pay system; potential for collusion between CLs and enumerators; lack of understanding of the importance of controls in place; and lack of resources responsible for financial administration of the program. Although efforts were clearly made to assess the impact of the change in payment from piece rate to hourly rate, these risks were not documented in the RMS.

Consequently, key financial risks were considered but not included in the formal risk management framework and did not benefit from the thoroughness of the Census RMS, where mitigation strategies would have been developed and communicated to staff, responsibility would have been assigned to a manager, and monitored for effectiveness.

Budgeting and Accountability

The Census Management Office (CMO) was responsible for the entire 2011 Census financial envelop. FOP was allocated approximately $130M for field operations and was responsible for assigning financial targets for Census field operations, and monitoring progress over program delivery, including total program spending. Census field operations were focused on attaining a response rate of 100% for Census and a sufficiently high response rate for NHS in order to ensure data quality.

A significant change occurred for the 2011 Census; enumerator pay was based on an hourly rate, rather than on piece rate. This change added complexity to financial planning of collection operations due to a lack of historical data. Regions were instructed to carry out field operations and report on program spending, with the understanding that initial financial targets were insufficient and that additional funds would be allocated to complete both Census and NHS collection operations, based on active monitoring of response rates.

Response rate targets for the NHS were communicated by CMO in July 2011. At that time, program costs in some regions were exceeding financial targets. Regions were being instructed by CMO to continue collection activities for Census and NHS until a satisfactory response rate across the country was achieved. Given the daily decision making required to achieve balanced response rates, regional management and the CMO tracked expenditure patterns and costs on a daily basis, but chose not to update the budgets on the same basis. Census management stated that costs were an integral part of the decision making process regarding follow up activities which were managed centrally with CMO, CRSB and Methodology Branch representatives.

The absence of historical data related to the way enumerators were paid, combined with the changes related to the new NHS, made it challenging for Census management to set informed financial targets for Census 2011. This vulnerability elevates the risk that regional management may have a diminished incentive to control NRFU costs for collections activities.

Field Management System & MIS Reports Used for Decision Making

Integrating performance targets throughout the reporting process yields better information for planning, forecasting and resource management. It also allows management to monitor actual performance against planned results and adjust its course if needed.

The FMS was developed following the 2006 Census to remediate problems related to the field operations in the areas of case distribution and management. The FMS's most critical functions were developed and tested; results were presented in the FMS Census Test Evaluation, in September 2009. They included the creation and management of NRFU assignments, pay entry and management features, access to and management of respondent action requests, access to management information, data capture of visitation records and the capture of collective dwelling information.

The FMS is a web based application that all field personnel could access using their own personal computer and Internet connection. It allowed field staff to have access to lists of dwellings requiring non-response follow-up, notification lists for questionnaires received at the Data Operations Centre, action requests, management information reports, and timely instructions and messages from management. Field staff also used it to input their productivity and to enter their pay and expense claims, on a daily basis.

The Management Information System (MIS) was the main source of information to monitor Census regional operations. While FMS was the system used to collect data on enumerators' productivity and time/expenses incurred, the MIS was the system used to compile results from FMS data. The majority of reports available in this data warehouse provided information on the progress of operations against set targets. Information on enumeration was entered in FMS, and pre-configured reports were accessible to supervisors and management at NHQ and in the regions through the MIS portal.

Reports which could be generated from this system provided information related to:

• Progress on recruitment, hiring and training hours,
• Cumulative number of hours claimed by enumerators,
• Progress on various activities of enumeration – including Early Enumeration and NRFU,
• Cumulative cost for each operation at the FOS and LCO levels, and
• Progress of shipments of completed paper forms.

MIS Summary reports were produced daily by ADs-Census for Regional Directors' review, and were used to monitor progress against operational targets set for the region, outliers trending above average on productivity and/or costs, and the percentage of allotted funds spent to date. However, these reports did not present cost items against any set financial targets. As an example, the Cost by Operation Key Indicator Report (KIR) provides the total daily and cumulative expenses, including: number hours and KMs incurred on a given day; the associated dollar value; other expenses; and the total amount of all expenses. These amounts are not compared to any target or expectation. The absence of historical data for remuneration based on an hourly rate precluded them from the benefit of using benchmarks or indicators as a basis to assess productivity. This impaired the ability to determine whether the cumulative amounts shown were within expectations or not, and explanations for budgetary deficits were not documented.

Field Operations Project (FOP) team was the bridge between regional operations and the Census program. FOP met with the regions on a daily basis via conference call to discuss results from MIS reports. FOP then met the Census Management Office (CMO) team on a weekly basis to provide updates on regional operations. The Operations MIS Committee, made up of all members of the Census Project Team, also reviewed MIS information as the operations unfolded to monitor the progress of the different projects. Although management reports were reviewed, key reports such as the Actual vs. Planned Costs and Cost by Operation KIR did not provide any explanation on the cause of financial variances or their impact on operations. This impaired management's ability to identify the source of problematic issues, take corrective actions, and possibly benefit from lessons learned in the future.

Overall, the FMS and MIS provided timely management information for case management and pay/expenses incurred by enumerators. The quality of information can be enhanced by identifying and explaining financial variances so that management can monitor actual performance against planned results and adjust its course as needed.

Recommendations:

It is recommended that the Assistant Chief Statistician of Census, Operations and Communications ensures that:

• Financial risks resulting from significant changes in the Census pay structure are included as part of the risk management framework;
• Financial and non-financial information gathered during the 2011 Census be used as benchmarks for planning and establishing targets at an appropriate level for the 2016 Census Program activities;
• The quality of information contained in MIS reports is enhanced to identify and explain financial variances so that management can monitor actual performance against planned results and adjust its course as needed.

Management Response:

Management agrees with the recommendations.

The Census of Population is the largest peace time operations in the federal government and by far the most complex program at Statistics Canada. The most challenging component of the program is the Field Operations, particularly the non-response follow up activity for millions of non-responding dwellings. Planning and executing non-response follow up activities requires a mature, dynamic and effective governance structure. Statistics Canada cannot determine at the outset of operations who will or will not respond and where, geographically, the non-responders will be located. Non-response activities are initially planned with response patterns from previous census cycles. The object of the field operations is to quickly, effectively and as efficiently as possible seek responses from those households that did not respond with initial contact.

Management must retain the flexibility to move its collection workforce and/or the workload around and to invest in follow up activities that ensure balanced response rates across the country. Based on years of experience conducting the Census, the Census Management Office, in collaboration with the Field Operations Project in Collection and Regional Services Branch determines productivity benchmarks and manage the financial risks in achieving a successful outcome at the corporate level.

In June 2010, a decision was taken by the government to maintain the 2011 Census of Population as a basic short-form census. All remaining questions that were previously asked in the long form census were included in a new voluntary survey, the National Household Survey. As a result, the complexity of field operations was augmented in 2011 with the introduction of the National Household Survey – the largest voluntary survey ever conducted by Statistics Canada.

Changes to the Census questionnaire and the introduction of the National Household Survey also introduced changes to the sample resulting in a significant increase in the number of dwellings receiving the Census and the National Household Survey questionnaires. The Census questionnaire was sent to 100% of Canadian dwellings – up from the planned 80% sample; with the National Household Survey distributed to 33% of Canadian dwellings – up from 20% sample for the long form Census.

In addition to initial funding received for the 2011 Census of Population, Statistics Canada received additional funding that was intended to be used for three purposes:

• Implement changes to the questionnaires and processing systems;
• Increased costs associated with the printing and mailing of additional questionnaires (both Census and the NHS)
• Increased follow up, if necessary, to ensure a 98% response rate on the Census

The additional funding was retained and managed by the Census Management Office. Approximately 50% of the fund was required to implement the changes and to fund additional printing and mailing costs. The remaining 50% was retained as a contingency and managed by the Census Management Office. The distribution of these funds was based on decisions taken by Policy Committee, on the recommendation of management of the Census Program. The recommendations were formulated based on daily discussions between Census Management Office, Collection and Regional Services Branch and Methodology Branch.

Given the daily decision making required to achieve balanced response across the country, it would be virtually impossible to update budgets for each Local Census Office on a daily basis. At the local level, managers are expected to manage productivity. However, expenditure control is an integral part of the decision making process to ensure there is continued return on investment. Expenditure patterns and costs are tracked on a daily basis by regional management in collaboration with the Census Management Office. Decisions on data collection priorities and budget allocations must be managed centrally with input from the Collection and Regional Services Branch, Methodology Branch and the Census Management Office.

The collection operations for the 2011 Census went extremely well and ultimately the contingency fund was not required to ensure the success of this collection activity. Achieving balanced response rates across the country then became the primary focus for the National Household Survey and the contingency funds were deployed for this purpose. Once it was determined that additional follow up would no longer improve the quality of the response rates, the decision to cease follow up activities was taken by Policy Committee and the remaining funds from the contingency were returned to the Treasury Board.

• The Census Management Office will ensure that financial risks related to the significant aspect of the census program will be included in Census risk management framework.
  
  Deliverables and Timeline: The risk register will be kept up-to-date for the 2016 program, on an on-going basis.
• As was done in the previous censuses, where data from the 2006 Census was used as benchmarks to support management of operations in 2011, the Census Management Office and the Research, Evaluation and Quality Management Project team will ensure that a similar approach will be developed for relevant operations for the 2016 program. The approach, while supporting sound financial management practices, must be flexible and targets established at appropriate levels to take into account the unpredictability of non-response operations Continuing the practice established in the 2011 collection operations, management from the Regional Offices, the Census Management Office and the Methodology Branch will work together to adjust daily operations priorities to achieve optimal response rates and to remain within allocated budgets.
  
  Deliverables and Timeline: A detailed management information approach will be developed and tested in-time for the 2016 operations.
• Census Management Office and the Field Operations Project team will ensure that based on the 2011 Census Program information, performance indicators/targets will be adjusted and financial MIS enhanced to provide better tools to improve the management of collection variable costs (e.g. alternative solutions will be considered to reduce kilometric costs).
  
  Deliverables and Timeline: A detailed management information approach will be developed and tested for the 2016 operations. This task will take place during the 2014 test period.

Objective No. 2: Adequacy and effectiveness of key controls embedded within the pay process surrounding payments to employees hired under the Statistics Act during the 2011 Census/NHS, including those managed through the field management system (FMS).

Stewardship

Training and directives with regard to cost control and the use of MIS reports identified roles and responsibilities but would have benefitted from clearer direction or guidance on the approach to carrying out oversight duties.

Planning and monitoring activities were mainly focused on meeting collection targets. CL and FOS reviewed claims and productivity reports for reasonableness. This key control, relied upon by Local Census Office Managers (LCOM) for approving pay and expense claims under Section 34 of the FAA, provided low assurance that costs were reasonable.

Results of audit tests performed demonstrated adherence to the Census Pay Process. All employees examined in the sample were deemed legitimate; all travel claims examined were pre-approved and signed by individuals with the proper authority; and payments were deposited in the bank accounts of the intended enumerators. Enumerator pay and expense claims were batched and approved by individuals holding authority under Sections 34 and 33 of FAA. However, in most cases, there was no evidence of any analysis or enquiries regarding accuracy and reasonableness of batches, as required for Section 34 sign-off. Consequently, individuals approving payments under Section 33 did not always have sufficient evidence that procedures required under Section 34 had been performed.

The approval process for payments to enumerators included two key controls which were relied upon in the Census pay process: CLs' daily review of submitted claims, and the review and analysis of productivity reports conducted by the CLs, FOSs, Pay Supervisors, and LCOMs before the pay and expense claims were approved for payment. To do so, reports obtained for the review and monitoring of pay and expense claims must yield timely and accurate information for each level of supervision to identify errors or instances of unreasonable claims submitted by enumerators.

Control over Enumerator Pay and Expenses

For the 2011 Census enumerators were paid an hourly rate to improve enumerator recruitment and retention. Consequently, timekeeping became an important element and was integrated into FMS. Enumerators used FMS to report their productivity and to enter their hours worked and expense claims on a daily basis.

Each CL was responsible for supervising the work of between 8 to14 enumerators. CLs were expected to review each pay and expense claim before recommending them for approval in the FMS. The system included an optional function, where enumerators could enter details or justifications in support of their claims. To support the CLs' monitoring activities, the FMS system also flagged the top 25% highest amount of TPC in a given LCO.

As a result, heavy reliance was placed on CLs to ensure the reasonableness of enumerator claims, by reviewing enumerators' performance prior to approving hours and KMs claimed and closely monitoring progress. Monitoring results and promptly addressing performance issues were key controls relied upon to ensure program objectives were met. Effective cost control procedures are critical to providing assurance to individuals with delegated authority for Sections 34 and 33 of the FAA prior to approving payments to enumerators.

Expectations in regards to cost control to be undertaken by CLs were outlined in the Directive 08: Cost Control and Financial Management for Field Operations, as follow:

"Crew leaders are expected to hold review sessions with their enumerators shortly after the start of each operation. They have access to G-forms and MIS reports to monitor productivity and time per case. They should be in contact with their enumerators daily. It is the crew leaders' responsibility to communicate expectations to their enumerators. Should production fall behind expectations or claims for kilometres be unreasonably high, closer supervision is needed, corrective action must be taken, and if needed, an employee can be replaced when necessary. Crew leaders should not hesitate to turn to their FOS for help when a problem cannot be solved."

This directive identifies roles and responsibilities for cost control, but would have benefitted from clearer direction or guidance on the approach. The document indicates that discussions should take place for non-performers; however it does not explicitly address a need for control over hours charged, and does not provide clear guidance on what is considered reasonable and what is not.

Throughout the Census collection period, the FMS also provided a variety of management reports and information, updated daily, and accessible to supervisors and program management for timely decision making. Productivity reports were deemed to be a key monitoring tool to be used by CLs, FOSs, AMFOs and LCOMs. The reports generated from the FMS data included information by geographic region, including TPC, number of attempts, KMs claimed, cases completed, CLD TPC average, LCO TPC average. However, procedures documents and training material did not explain how to use the MIS reports in performing oversight duties for cost accuracy and reasonableness.

Through interviews and testing, the audit team assessed the CLs' review processes pertaining to pay/expenses and productivity reports, in order to determine their effectiveness in providing assurance that costs were controlled. Our examination of approximately 50 claims revealed that nearly two thirds of the claims did not include comments to explain hours/KMs, even in instances, for example, where more than 100 KMs were claimed and no questionnaires were collected.

The lack of justifications places the onus on the CL's to follow up with enumerators who were automatically flagged by the FMS as under performers. Furthermore, CLs interviewed stated that when they did contact enumerators to enquire about their time and expenses, they rarely denied claims because the enumerators were able to provide various plausible explanations (e.g. excessive traffic congestion, road construction, etc,) to justify claims that appeared excessive. In most cases, CLs were not able to challenge exaggerated claims, only those which appeared grossly inflated were investigated or received attention.

As mentioned above, the FMS automatically flagged high amounts of TPC, and generated a report which identified the top 25% highest amount of TPC in a given LCO. CLs had access to monitoring reports and were expected to use this information prior to recommending pay claims. The control was meant to quickly identify claims that appeared unreasonable, in comparison to other enumerators for the same area. Tests revealed that if an enumerator did not report any forms collected, hours charged trying to collect questionnaire would not be flagged by the system's reports. The system did not flag cases of high TPC until the questionnaires were collected within the collection unit.

After examination and interviews with regional staff, the audit team noted that enumerator's time and expenses vs. questionnaire information were not always uploaded into MIS simultaneously. Depending on the time of day at which the enumerators input their time into FMS, loading the case information did not always occur on the same day as the time and expense information was entered. This situation impacted the accuracy of TPC statistics for collection units, providing varying results. Some CLs interviewed mentioned that they did not always follow-up on flagged cases, as in many cases they noticed that the numbers eventually evened out. This affected the timeliness and integrity of information used to identify poor performers and reduced the effectiveness of the control. Census Evaluation results indicated that 95% of all claims were paid without any edits by CLs, demonstrating that few changes resulted from the CLs' monitoring of claims.

Field operations supervisors (FOS) are the front line supervisors in comparing productivity to cost by using the MIS reports related to each operation. Their other main role was to monitor reports from their crew leaders and provide guidance when needed. FOS supervised between 7-12 CL's, all working in different geographical locations and contexts. Interviews with FOSs revealed that they did not monitor the effectiveness of reviews conducted at the CLs' level and were mainly focussed on optimizing staff distribution according to productivity levels attained. Most importantly, they stated that the expectation that FOSs would undertake this analysis was not realistic considering the short timeframe for NRFU.

Audit testing showed that enumerators flagged by the system for low productivity remained active while no significant improvement occurred. There was no evidence that follow-up had been performed for any of these poor performers. Conversely, performance evaluation for these same enumerators indicated "met" or "exceeded expectations" – by making reference to their ability to collect questionnaires, never commenting on their productivity or value for money.

The audit revealed that planning and monitoring activities were mainly focused on meeting collection targets. The review of FMS claims and productivity reports by Crew Leader and Field Operation Supervisors, considered a key control relied upon by Local Census Office Managers (LCOM) for approving pay and expense claims under Section 34 of the FAA, provided low assurance that costs were reasonable.

Adherence to Census Pay Process

Authority for sign-off under Section 34 of the FAA for fees paid to enumerators resided with the LCOM. It was expected that procedures outlined in the Form 49-E Financial Office Manual would be followed and executed.

Once CLs recommended enumerators' claims, a summary of claims (batches) was prepared within each LCO for Section 34 approval by the LCOM, after which it was sent to the regional finance group and respective Pay Units for Section 33 approval. Details of the pay process in place during the 2011 Census can be found in Appendix B.

Attributes from Form 49-E Financial Office Manual and the Guide for Local Census Office Managers were used to develop our audit procedures. Testing was to provide assurance that controls were in place to ensure that:

• Enumerators who received payments were legitimate and were hired according to established procedures;
• Section 32 of FAA approval for Travel Authorization Forms was documented and supported;
• Section 34 of FAA approval was documented and supported;
• Section 33 of FAA approval was documented and supported; and
• Payments were made to the bank account of the intended enumerator.

A sample of 25 enumerators' pay claims were selected by the audit team for each region, covering two pay periods within the peak time of NRFU. Ten Crew Leader Districts (CLDs) were randomly selected within these regions, including enumerators that had claimed a significant number of hours. The sample included enumerators with a Time per Case (TPC) within the average and those exceeding the average.

The audit revealed the following:

In accordance with program directives, employee files were uploaded from the original application files created in the Census Automated Recruitment System (CARS), into FMS. In compliance with Form 46E - Directive 14: Recruitment for the 2011 Census, employees' application forms were subjected to effective segregation of duties throughout the approval process prior to being hired. Applicant records were traced back to CARS and examined to ensure that each level of approval was approved electronically by the right persons. Multiple-level approval processes are an effective measure in preventing the creation of fictitious employees. Original application forms, and other required documentation, which includes employee photos, were also on file. All employees examined in the sample were deemed to be legitimate employees.

Travel claims submitted by enumerators were pre-approved prior to travel. Travel Authority Request forms were signed by individuals having delegated authority under Section 32 of the FAA.

Enumerator pay and expense claims were batched and approved by individuals holding authority under Sections 34 and 33 of FAA. However, in most cases, there was no evidence of any analysis or enquiries regarding accuracy and reasonableness of batches, as required for Section 34 sign-off. Consequently, individuals approving payments under Section 33 did not always have sufficient evidence that procedures required under Section 34 had been performed.

Pay supervisors had the ability to modify employee pay account information in FMS. Testing found that payments were made to the bank accounts of intended enumerators. Prescribed forms required for amending pay account information for enumerators were on file and all required signatures matched.

Recommendations:

It is recommended that the Assistant Chief Statistician of Census, Operations and Communications ensures that:

• Training and directives with regard to cost control and the use of MIS reports are revised to provide clear direction and guidance on the approach to carrying out oversight duties.
• Key controls relied upon for approving pay and expense claims under Section 34 of the FAA, are reviewed for feasibility and effectiveness, taking into consideration the short time frame for carrying out NRFU activities;
• Procedures required under Section 34 of the FAA are consistently performed, and yield sufficient evidence and assurance for approval of payments under Section 33.

Management Response:

Management agrees with the recommendations.

• The Field Operations Project team will ensure that performance indicators/targets will be adjusted and processes refined to ensure managers have the tools they require to better manage collection variable costs. Training and directives on the use of MIS reports will be updated and responsibilities will be clarified to ensure managers understand and comply with their financial obligations.
  
  Deliverables and Timeline: A detailed management information approach will be developed and tested for the 2016 operations, including training for those involved. This task will take place during the 2014 test period.
• The Field Operations Project team will ensure that Section 34 authorities will be reviewed and adjusted as required. Key control processes will be clarified and appropriate training provided to ensure those individuals approving expenses under section 34 comply with their delegated authorities.
  
  Deliverables and Timeline: A detailed management information approach is developed and tested for the 2016 operations, including relevant information in relation to FAA obligations. This task will take place during the 2014 test period.
• The Field Operations team will ensure that Section 34 authorities will be reviewed and adjusted as required. Procedures required under Section 34 of the FAA will be clarified and appropriate training provided. In collaboration with the Finance Branch, individuals approving pay and expense claims will be monitored for compliance with their delegated authorities.
  
  Deliverables and Timeline: A detailed management information approach will be developed and tested for the 2016 operations, including relevant information in relation to FAA obligations. This task will take place during the 2014 test period.

Appendices

Appendix A: 2011 Census Field Structure

Appendix B: Pay Process per Form 49-E Finance Office Manual

Chapter 3 - Filing

1. Unless otherwise specified, standard regional office filing procedures will be followed.
2. Statistics Act employees will have one file for all their documentation (e.g., appointment and pay forms). These files will be maintained in alphabetical order in the LCO within the AMA / Pay Unit. All account forms, hiring forms and support documents will be retained.
3. Each Statistics Act employee file should contain:
   1. Hiring Checklist – Statistics Act Staff, Form 21B
   2. Statistics Act Candidate Screening Form, Form 22A
   3. Recommendation and Assignment of Statistics Act Staff, Form 23
   4. Oath or Affirmation of Office and Secrecy, Form 24
   5. Implication of the Oath or Affirmation, Form 24A
   6. Terms and Conditions of Employment (appropriate Form 25 series)
   7. Census Application for Employment, Form 26 (if application was not completed online), resume, CV and any other supporting notes produced during the selection process
   8. Selection Test, Form 27 or 27B / PSC Answer Sheet, Form PSC2256
   9. Non-supervisory Candidate Assessment Booklet, Form 27C/Answer Key – Selection Test, Form 27D
   10. Census Reliability Assessment, Form 26B
   11. FMS User Account Agreement Form, Form 26E
   12. Change of Address Form, Form 30B (if applicable)
   13. one photo
   14. Recommended Weekly Hours and Expenses Log, Form 32 (only needed when expenses have been claimed or when hours and productivity are entered by proxy)
   15. receipts are to be attached and filed with the Form 32
   16. Performance Evaluation, Form 38
   17. Direct Deposit Information, Form 39
   18. Deduction Declaration, Form 39B, applicable TD1, TP-1015.3-V and/or TD1-IN forms
   19. language test (if applicable)
   20. other relevant documentation (correspondence, notes of disciplinary meetings, etc.).

The contents of these forms are highly confidential and these forms must be filed in a secure, lockable cabinet with limited access, which must be kept locked after hours and when not in use.

4. Files should be created on an ongoing basis and filing should be kept up to date in order to allow staff to locate documents quickly and easily.
5. Batch payment authorization reports, payment confirmation report (CDFS) and summary report (CDFS) should be filed by date and in batch number sequence.
6. The filing system must be maintained so that an audit trail can be followed by individual, by Batch Payment Authorization Report, or by Payment Confirmation Report.
7. Form 31 and other documents preauthorizing expenditures will be sent to the regional census centre (RCC) for payment with a copy being retained and filed in the LCO where it originated. These files will be maintained in province, census district, and collection unit code sequence to enable easy access should they need to be retrieved at a later date.
8. All protected information collected must be stored in a locked cabinet. These cabinets must be kept locked, especially at the end of each day, with the keys or combination stored in a secure and central location. If the need to store large volumes of protected material arises, contact Departmental Security.

Appendix C: Audit Criteria

Appendix C: Audit Criteria
Table summary
The table in Appendix C identifies the Audit Criteria, controls (existing and expected), as well as the policy instrument used as the source of these criteria.

Line of Enquiry / Core Controls / Criteria	Controls (existing and expected)	Policy Instrument
1) Adequacy of the control framework in place surrounding financial planning, monitoring and reporting of expenses processed through FMS.
Risk Management
1.1 Management identifies the risks that may preclude the achievement of its objectives, and assesses the existing controls in place. (RM-2 and 3)	1.1.1 Management is aware of the potential impacts of changes in the Census structure (NHS), as well as significant operational changes (i.e. hourly pay) and has tools in place to manage these risks.	Management Accountability Framework RMS User Guide Census The 2011 CensusManagement Guide Statistics Act Form 46-E Directive 08: Cost Control and Financial Management for Field Operations
Budgeting and Accountability / FMS & MIS Reports Used for Decision Making
1.2 Financial management policies and directives are appropriately designed for Census field operations to ensure that budgeted and actual cost are reported and monitored at the program level. (ST-5) Reviews are conducted to analyze, compare and explain financial variances between actual and plan. (ST-15)	1.2.1 Guidance is provided by Finance on the financial information structure, available reporting tools and financial management practices that need to be followed for the Census.	Management Accountability Framework MIS Portal Suite
	1.2.2 Reviews and analysis are performed over financial results and is compared to budgets in order to ensure that regional operations are managed appropriately.	Management Accountability Framework MIS Portal Suite
	1.2.3 Reports are prepared and communicated to relevant stakeholders on results of regional operations.	Management Accountability Framework MIS Portal Suite
2) Adequacy and effectiveness of key controls embedded within the pay process surrounding payments to employees hired under the Statistics Act during the 2011 Census/NHS, including those managed through the Field Management System (FMS).
Control over Enumerator Pay and Expenses
2.1 Information used to approve enumerator claims is accurate and timely. Productivity reports are reviewed and analyzed by FOSs, Pay Supervisors, and LCOMs before the related pay and expense claims are approved for payment accuracy and timeliness. (ST-10/18)	2.1.1 Review and monitoring of performance is conducted on a daily basis and poor performers are identified and are dealt with in a timely fashion.	Management Accountability Framework Financial Administration Act Analyzing NFRU Enumerator Productivity KIR Supervisor FMS User Guide and Forms Statistics Act
	2.1.2 Prior to recommending claims, supervisors either recommend the claims or consult with the employee to clarify and/or correct any discrepancies, as necessary.	Management Accountability Framework Financial Administration Act Analyzing NFRU Enumerator Productivity KIR Supervisor FMS User Guide and Forms Statistics Act
2.2 Reports obtained for the review and monitoring of pay and expense claims could identify errors or cases of unreasonable claims submitted by enumerators. (ST-20)	2.2.1 Claims are reviewed by supervisors and approved accordingly.	Management Accountability Framework Manual 46.39 Recommending Claims and Authorizing Payments by the FOS and LCOM
2.3 Employees were provided with necessary training, tools, resources and information to support the discharge of their monitoring and oversight responsibilities. (PPL-4)	2.3.1 Training for employees with supervisory functions in field operations provides direction on oversight over financial activities under their responsibilities i.e. review of pay and expense claims, productivity, S. 34 and S. 33.	Management Accountability Framework E-Library - en.STCwiki Statistics Act
Adherence to Census Pay Process
2.4 Pay and expense claims are reviewed for accuracy and reasonableness and approved by the appropriate individual (S. 34 of FAA and S. 33 of FAA). (ST-15/18)	2.4.1 Section 34 of FAA review, and approval is documented and based on information allowing the identification of potential errors.	Management Accountability Framework Managers Financial Signing Authorities and the Procurement Process for Census Field Financial Administration Act Form 49E - Finance Office Manual Section 33 - Census Regional Finance Officers Form 32 Directive
	2.4.2 Segregation of duties for S. 33 of FAA is in place and monitored by FMOSD.	Management Accountability Framework Managers Financial Signing Authorities and the Procurement Process for Census Field Financial Administration Act Form 49E - Finance Office Manual Section 33 - Census Regional Finance Officers Form 32 Directive Form 46E - Directive 14: Recruitment for the 2011

Appendix D: Acronyms

Appendix D: Acronyms

Acronym	Description
AMFO	Assistant-Manager Field Operations
CARS	Census Automated Recruitment System
CL	Crew Leader
CLD	Crew Leader District
CMO	Census Management Office
CPT	Census Project Team
CRSB	Collections and Regional Services Branch
DPC	Data Processing Center
FAA	Financial Administration Act
FMS	Field Management System
FOP	Field Operations Project
FOS	Field Operations Supervisor
HQ	Headquarters
IIA	Institute of Internal Auditors
KM	Kilometres
LCO	Local Census Office
LCOM	Local Census Office Manager
LOE	Line of Enquiry
MAF	Management Accountability Framework
MIS	Management Information System
NHS	National Household Survey
NRFU	Non-Response Follow-Up
PSEA	Public Service Employment Act
RCC	Regional Census Centre
RMD	Resource Management Division
RMS	Risk Management System
TBS	Treasury Board Secretariat
TPC	Time Per Case

Linking Data for the Survey of Household Survey Spending to Income Tax Records File (T1)
2006 Farm Environmental Management Survey (FEMS): Linkage to the 2006 Census of Agriculture
Analysis of Business Behaviour
Commercial and Institutional Consumption of Energy Survey (CICES) Data Linkage: 2007 CICES
Small– and Medium–sized Enterprise Statistics (SMEStats) Data Warehouse
Omnibus Record Linkage Authority for Economic Statistics Programs
Socioeconomic Influences on the Use of Physician Services in Ontario
Self-perceived Unmet Need and Use of Health Care in Ontario
Linking Telephone Numbers to the Living in Canada Survey- Pilot Sample File
Living in Canada Survey-Pilot: Personal Income Tax Files and Pension Plans in Canada File Linkages
Census of Agriculture: Feasibility Study of Linkage to Tax Data for Replacement of Farm Financial Questions
Using 2006 Census of Population Data to Improve the Quality of the Estimates for the Access and Support to Education and Training Survey
Evaluation of the Census Internet Response Option: Linking 2006 Census Data to the Evaluation Surveys Data
Longitudinal Immigration Database and Immigrant Information in the Longitudinal Administrative Databank: 2005 to 2009 Updates
Omnibus Record Linkage Authority for Improving the Population and Household Survey Programs

---

Linking Data for the Survey of Household Survey Spending to Income Tax Records File (T1)

Purpose: The purpose of this linkage is to obtain income data and reduce respondent burden, interview time and collection costs for the Survey of Household Spending.  Carried out on a voluntary basis, the Survey of Household Spending gathers detailed information amounts spent on food, clothing, shelter, transportation, health care and other items in order to understand spending habits of households in Canada.  Information is also collected about dwelling characteristics and household furnishings and equipment.

The linkage allows obtaining information on income variables without burdening respondents with detailed questions about their income.  The income data is important for the Survey of Household Spending and allows analysis of the relationship between spending and income. This survey is the only Statistics Canada survey that releases information on this relationship.

Information from the Survey of Household Spending is used in the development of the Consumer Prix Index, in the System of National Accounts and is widely used in developing various federal and provincial policies and programs.

Description: The Survey of Household Spending databaseand the T1 File will be linked using the address, city, date of birth, first name, surname, sex, province, NYSIIS and SNDX code for surname, postal code, marital status, telephone number and first initial. This information will be removed from the linked file as soon as the linkage is completed, and stored separately. Access to these files will be restricted to Statistics Canada employees whose assigned work activities require access.

Output: No information containing personal identifiers will be released outside of Statistics Canada from this linkage activity. Only aggregate statistics and analysis conforming to the confidentiality provisions of the Statistics Act will be released outside of Statistics Canada.

---

2006 Farm Environmental Management Survey (FEMS): Linkage to the 2006 Census of Agriculture

Purpose: To provide current information on national agri-environmental conditions and risks and to improve our understanding of on-farm environment management practices. While the FEMS data themselves can be used to meet this objective, the linked file permits a broader range of analytical studies.

Findings will aid in understanding the uptake of beneficial management practices (BMPs) and under what conditions they are being implemented. The BMPs are practices scientifically proven to reduce the impact of agricultural activities on soil and water resources while maintaining the economic viability of the industry.

The linked data are a key input into Agriculture and Agri-Food Canada’s National Agri-Environmental Health Analysis and Reporting Program (NAHARP) which, through models, provides the agriculture industry, decision-makers and the Canadian public with information on the environmental performance of Canadian agriculture. Findings may influence industry initiatives as well as federal and provincial programs and policies aimed at encouraging the sustainable agri-environmental management of air, land, manure, biodiversity and water.

The linked file will contribute to the reduction of response burden; certain important information items collected on the Census of Agriculture were deliberately left off the FEMS questionnaire. Thus, the FEMS was designed to be used along with important information collected on the Census of Agriculture

Description: To obtain information on farm operations (i.e., area farmed, number of livestock, expenditures for fertilizers, herbicides, insecticides and fungicides) and socio-economic factors, the 2006 FEMS will be linked to the 2006 Census of Agriculture. Only records for which consent was given by the 2006 FEMS respondents will be linked. The files will be linked using the Farm Register number and will not contain any direct identifiers such as name, address or telephone number.

Output: Only aggregate data that conform to the confidentiality provisions of the Statistics Act will be released outside of Statistics Canada. Findings will be published in Statistics Canada’s “Agriculture and Rural Working Paper Series” (Catalogue no. 21-601-M). Aggregate statistics at the eco-region/farm type/province cross-classified levels will be produced for Agriculture and Agri-Food Canada. The linked file, without identifiers, will be retained indefinitely.

---

Analysis of Business Behaviour

Purpose: In order to improve the business environment and thereby enhance competitive advantage, policy makers need better information on the behaviour of businesses across the economy and over time. Administrative databases are being used to support this analysis without increasing reporting burden. These analyses will examine changes in employment, research and development expenditure, value of exports and other corporate financial data as part of a bigger project to look at business strategies and their outcomes over time. As businesses will be followed over time, changes in their characteristics will be monitored and examined in relation to changes in government programs, such as taxation, support for entrepreneurial activity, knowledge transfer, and opportunity for the highly skilled. This work will lead to the development of a longitudinal business panel survey that will be used by policy departments, such as Industry Canada, to develop government policies in a range of areas.

Description: This project links the Business Register (BR), the Longitudinal Employment Analysis Program (BR, LEAP), the Research and Development in Canadian Industry Survey, the Exporter Register and the General Index of Financial Information (GIFI) databases. The LEAP database will provide a longitudinal source of information on employment, salaries, births and deaths of enterprises. A deterministic record linkage procedure will be used to link the BR, LEAP, RDCI, Exporter Registry and GIFI datasets. The Business Number (BN) and the Business Register Identification Number (BRID) will be used as the key identifiers. The databases will be linked, starting with the 2001 data year to the 2005 data year.

Output: Only aggregate data conforming to the confidentiality provisions of the Statistics Act will be released outside of Statistics Canada. Reports will be published in the Working Paper series of the Science, Innovation and Electronic Information Division (SIEID) at Statistics Canada and an article highlighting the key findings will be published in the Innovation Analysis Bulletin of SIEID. The linked files will be retained until January 2011, at which time they will be destroyed. The linkage keys, such as BN and BRID, will be removed from the linked file and stored separately. This will allow future linkages, but removes the possibility of direct identification of a business by analysts. The linkage keys will also be destroyed by January 2011.

---

Commercial and Institutional Consumption of Energy Survey (CICES) Data Linkage: 2007 CICES

Purpose: The purpose of the survey is to analyze energy consumption patterns and assess how well Canada is fulfilling its commitments to increasing energy efficiency and reducing greenhouse gas emissions that contribute to climate change. These data will be used to develop and refine programs available to help businesses, institutions and organizations increase their own energy efficiency and reduce greenhouse gas emissions. To increase the quality of the survey data, energy consumption information from a knowledgeable third party is linked to the survey data when the primary respondent cannot provide this information directly.

Description: Survey respondents who cannot supply the energy data directly provide written consent to Statistics Canada to obtain their energy consumption data from a knowledgeable third party (such as their landlord or property manager), and for their data to be combined with the energy consumption data provided by the third party. The third–party contact identified by the respondent also provides written consent to combine the energy consumption data they provide with the data provided by the respondent.

This linkage covers the survey conducted for the 2007 reference year.  Linkage is only done when both the primary respondent and the third party consent.

Output: The resulting linked file, stripped of identifiers, will be retained by Statistics Canada for five years until December 2013. Information from respondents and third parties who consent both to linking and to sharing of their data will be provided, without identifiers, to Natural Resources Canada and Environment Canada, who will keep this information confidential and for an indefinite period.

---

Small– and Medium–sized Enterprise Statistics (SMEStats) Data Warehouse

Purpose: Small businesses have long been recognized as a key component of the Canadian economy. The SMEStats Data Warehouse will profile small– and medium–sized enterprises (SMEs) in Canada by making available key demographic statistics and performance indicators. Information of this kind is essential for policymakers to identify sectors and regions in distress and in the formulation and evaluation of tax and other policies as they impact on small– and medium–sized business in Canada. Some important policy questions this data warehouse will be used to answer include:

• What is the continuing role of SME businesses in economic and employment growth at the provincial and sub–provincial level? In what regions and sectors are pressures on SMEs emerging?
• What are the attributes and characteristics of the most successful of the SME businesses, the so–called “gazelles”?
• How do SMEs compare with other businesses in terms of profit and profitability, cost structures and productivity?
• What is the role of SMEs in job and revenue creation?
• Do SMEs take equal or relative advantage of tax credits offered to all businesses, for example, credits intended to improve the energy efficiency of businesses?
• Is globalization contributing to convergent or divergent growth among businesses? Is there a growing divide between large and small businesses?
• How do Canadian businesses perform in comparison to businesses in other countries, based on internationally–accepted performance indicators?

Description: The SMEStats Data Warehouse is a cross–sectional and longitudinal linkage of Statistics Canada’s Business Register, the Longitudinal Employment Analysis Program (LEAP), Payroll Deductions file (PD7), and income and balance sheet information from T1 and T2 tax returns filed by businesses with Canada Revenue Agency, on an annual basis, for data years 2000 to 2007. This information is linked using the Business Number (BN), Statistical Enterprise Number (SNUM) and Legal/Operating Name and Postal Code.

Output: Only aggregate data conforming to the confidentiality provisions of the Statistics Act will be released outside of Statistics Canada. A brief methodology report will also be prepared explaining the database creation processes, constraints and key issues related to the quality of the data and comparisons with other data.

Outputs will include SME profiles consisting of counts of enterprises by employment and revenue size grouping by industry (NAICS) and geography down to fine sub–provincial areas; and age of business. SME performance indicators will focus on issues related to firms, employment and wealth. Additional ad hoc analyses may also take place. The linked files will be retained until January 2011, at which time they will be destroyed.

---

Omnibus Record Linkage Authority for Economic Statistics Programs

Purpose: To facilitate the efficient and effective production of high-quality statistical outputs from Statistics Canada’s economic statistics programs.

Statistics Canada’s strategy for its economic statistics programs is heavily reliant on the mix of survey data and administrative data. This approach yields higher quality statistics, for less cost and with less burden on Canadian businesses, as information reported for administrative purposes to public or private organizations is also used by Statistics Canada for statistical purposes.

Record linkages can be for three purposes:

• Data production:  Use already collected data for a statistical program rather than re-collect.  It also includes frame creation, preparation of contact material, imputation for item non-response.
• Analysis to support production:  Analysis for purposes of data certification and data quality evaluation, such as evaluating trends in one data set by examining the reports of the same businesses in another data set.
• Analysis to provide information:  Making use of combined datasets to support analysis that is not possible by a single existing data set. In most cases, the outputs from these projects are officially released.

Statistics Canada has adopted this strategy following consultation with a wide range of businesses.  Generally, record linkages are approved on a project-by-project basis.  However, given the fundamental importance of linkage in the economic statistics programs, linkages are approved under this omnibus authority, subject to the following considerations:

• No linkage should damage Statistics Canada’s relationship with respondent businesses.
• No privacy-invasive linkages shall be carried out without a demonstrated public good. Privacy concerns are present when information from unincorporated businesses is linked.
• If the linkage involves client-specified group of businesses, no files are to be linked if the results might harm the interests of that group.

The following are excluded from the umbrella of the omnibus record linkage authority:

• Linkages that would have significant privacy implications, including all activities of the Agriculture Division, and targeted linkages related to small and/or unincorporated businesses
• Linkages that involve client-supplied lists of businesses
• Linkages that present a significant risk of residual disclosure

Description: For a particular program, linkages can be one or more of the following:

• Survey data to administrative data for the same time period.
• Survey data for a specific time period to the same survey data from a different time period.
• Survey data for a specific time period to data from another survey for the same time period.
• Administrative data to another administrative file.

In this manner, both cross-sectional and longitudinal linkages are possible.

Survey data refer to data collected directly from businesses by Statistics Canada.  Administrative data refer to data collected by other organizations as part of their mandate.  Statistics Canada uses administrative data for statistical purposes.  Although not the only such files, the primary administrative files used for these purposes are tax files collected by the Canada Revenue Agency.

Usually, linkage is conducted using Business Numbers and other similar identifiers.

Output: All outputs will conform to the legal/confidentiality requirements of the Statistics Act, and any other federal legislation that may apply. Retention periods for linked files vary from project to project. Files that are produced for survey-production purposes generally have indefinite retention.  On the other hand, files linked for a specific project will be retained only until the end of the project. Direct identifiers (such as names, addresses) are usually not be maintained on linked files, but will be maintained separately. As of July 1, 2008, Statistics Canada will maintain an inventory of all linkages conducted under this omnibus authority.

---

Socioeconomic Influences on the Use of Physician Services in Ontario

Purpose: To examine whether the socioeconomic status of patients influences their use of general practitioner (GP) services, their referral patterns to specialists, and their joint use of different physicians’ services. Actual, rather than self-reported, measures of physician utilization will be employed. More specifically, the research will:

1. assess whether use of GP services, measured by the number of visits, the type of services and the related expenditures, varies with  patients’ socioeconomic status (measured by income and education), after taking health care needs (measured by self-reported health status) into account;
2. model the pathway between patients’ use of GP services and their use of specialist services, taking into account socioeconomic status differences and health care needs; and,
3. determine whether certain categories of survey respondents systematically over- or under-report the number of physician visits they made in the year prior to the survey.

This study is part of a PhD candidate’s dissertation and findings will provide valuable information to the medical community and to health policy makers. The results may indicate that certain population groups are disadvantaged by the current delivery of health care, in which case the study will indicate where changes should be made

Description: Data from respondents to the Canadian Community Health Survey (CCHS), cycle 1.1 (2000-2001) will be linked to administrative information on visits to physicians and health services received in Ontario. Only those cases where informed consent was received from survey respondents will be linked.

The administrative databases used in this research project are the Medical Services files – based on Ontario Health Insurance Program (OHIP) claims for three fiscal years: 1999-2000, 2000-2001, and 2001-2002.

The data will be linked using deterministic matching on an encrypted health number. A validation procedure, carried out by the Ontario Ministry of Health and Long-Term Care (MOHLTC), has made sure only valid health card numbers for CCHS records are found on the cohort file, and are encrypted. Health card numbers have been similarly encrypted on the health administrative databases. Personal identifiers will be removed from the files and a number assigned by Statistics Canada will be appended to records in the administrative databases that link to CCHS records.

The study is part of a pilot project between Statistics Canada, the MOHLTC and McMaster University, aimed at enhancing access to Ontario health information by the research community. The creation of an analytical file, as well as aggregation and analysis of the data, will be carried out in the Statistics Canada Research Data Centre (RDC) at McMaster University. The researcher accessing the data in the RDC will do so as a deemed employee of Statistics Canada.

Output: Only aggregate statistical outputs conforming to the confidentiality provisions of the Statistics Act will be released outside of Statistics Canada. Research findings will be disseminated through a release in The Daily, research papers, peer-reviewed journal articles or through presentations at national or international conferences. Statistics Canada will retain the linked files for a period of five years, that is, until September 2013, at which time they will be destroyed.

---

Self-perceived Unmet Need and Use of Health Care in Ontario

Purpose: The study will show what types of Ontario residents experience unmet health care needs, and examine the relationship between the reasons for the unmet need and the use of health care services. The overall objective of this study is to characterize how the use of physician and hospital care by those who report an unmet health care need compares to similar individuals who do not report an unmet health care need.

The relationship between reporting an unmet health care need and overall physician and hospital use will be examined. As well, the relationship between the reason for the unmet need, that is, due to personal choice, to system barriers, to wait times, or for other reasons, and the amount and type of physician or hospital services received will also be examined. Findings may be used by the health care system to better focus its resources to address needs that come from system limitations.

Description: Data from respondents to the Canadian Community Health Survey (CCHS), cycle 1.1 (2000/2001) will be linked to administrative information on diagnoses, health care services received, and expenditures for those respondents who visited physicians, stayed in hospitals or underwent outpatient procedures in Ontario hospitals. Only those cases where informed consent was received from survey respondents will be linked.

Two administrative databases are used in this research project: 1) the Medical Services files – based on Ontario Health Insurance Program (OHIP) claims for 1999/2000, 2000/2001 and 2001/2002; and the Discharge Abstract Database (DAD) inpatients and day procedures for 1999/2000, 2000/2001 and 2001/2002.

The data will be linked using deterministic matching on an encrypted health number. A validation procedure, carried out by the Ontario Ministry of Health and Long-Term Care (MOHLTC), has made sure only valid health card numbers for CCHS records are found on the cohort file, and are encrypted. Health card numbers have been similarly encrypted on the health administrative data files. Personal identifiers will be removed from the files and a number assigned by Statistics Canada will be appended to records in the administrative databases that link to CCHS records.

The study is part of a pilot project between Statistics Canada, the Ontario MOHLTC and McMaster University, aimed at enhancing access to Ontario health information by the research community. The creation of an analytical file, as well as aggregation and analysis of the data, will be carried out in the Statistics Canada Research Data Centre (RDC) at McMaster University. Researchers accessing the data in the RDC will do so as deemed employees of Statistics Canada.

Output: Only aggregate statistical outputs conforming to the confidentiality provisions of the Statistics Act will be released outside of Statistics Canada. Research findings will be disseminated through a release in The Daily, working papers, articles in peer-reviewed journals and presentations at national and international conferences. Statistics Canada will retain the linked files for a period of five years, that is, until September 2013, at which time they will be destroyed.

---

Linking Telephone Numbers to the Living in Canada Survey- Pilot Sample File

Purpose: To improve response rates and population coverage for the pilot survey and to reduce costs.

The Living in Canada Survey-Pilot (LCS-Pilot) is a longitudinal survey that will test the survey content of the Living in Canada Survey (LCS). The LCS is Canada’s first comprehensive long-run socio-economic panel survey of households. The information collected will serve as the basis for a better understanding of Canadian society in the 21st century.  It emphasizes the relationships between four major domains of people’s lives—work, family, education and health—and allows for research in areas important to Canadians. Moreover, the results of this survey will be used by different levels of government in order to help them develop better policies and programs. Surveys such as this one have already proven to be valuable in other countries, enabling Canada to compare itself internationally on many important issues.

This data linkage activity will associate telephone numbers with the addresses of dwellings selected for the LCS-Pilot sample. These linked telephone numbers will be used during collection activities to contact households. This linkage activity will facilitate contact, improve response rates and reduce interviewing costs for the LCS-Pilot.

Description: The sample file for the LCS-Pilot will be linked to local telephone company files (also called billing files) and to the InfoDirect database. Addresses (including street number, street name, unit number, city, province and postal code) will be matched on these files, and the telephone number will be extracted to facilitate contact with dwellings. Only publicly available telephone numbers will be used in this linkage activity. This linkage will be performed once in the fall of 2008.

Output: The output from this linkage activity will only be used within Statistics Canada for collection purposes for the LCS-Pilot. Results of this linkage will be retained for the life of the LCS-Pilot field period (up to 2014 in the current plans) to facilitate contact with households in subsequent waves of this panel survey.

---

Living in Canada Survey-Pilot: Personal Income Tax Files and Pension Plans in Canada File Linkages

Purpose: To improve the quality of the data collected for the survey and to reduce response burden and survey costs.

The Living in Canada Survey-Pilot (LCS-Pilot) is a voluntary longitudinal survey that will test the survey content and methodology of the Living in Canada Survey (LCS). The LCS is Canada’s first comprehensive long-run socio-economic panel survey of households. It emphasizes the relationships between four major domains of people’s lives—work, family, education and health—and allows for research in areas important to Canadians. Moreover, the results of this survey will be used by different levels of government in order to help them develop better policies and programs. Surveys such as this one have already proven to be valuable in other countries, enabling Canada to compare itself internationally on many important issues.

The linkages will allow for the collection of some variables that are difficult for respondents to remember accurately, such as pension plan contributions, or income taxes paid. The linkages will add retrospective income data which complements retrospective education, family and work data already collected in the survey.

The linkages will also replace more than 20 income questions on the survey. Respondents are asked to report detailed information on income from earnings, investments, government sources, and pensions. To shorten the interview and reduce response burden, respondents are given the option to grant Statistics Canada permission to access their tax records for the duration they remain within the survey. The linkage will also reduce response burden to the survey by allowing use of high quality administrative data on income, earnings and employers for past years, which eliminates the need to ask these in separate questions.

In addition, the resulting survey data will be of higher quality as administrative income data has been judged to be of better quality than self-reported income.

Description: Data linkages will only be made for those respondents who give Statistics Canada their consent to access their tax files. For these respondents, their income, earnings and employer information will be retrieved from the T1 tax file, and the T4 Summary and Supplementary files. To find the individual’s Social Insurance Number (SIN), the respondents first and last name, sex, date of birth, marital status, address and postal code, the presence of the respondent’s spouse in the household and the date of birth of the respondent’s spouse (if the respondent’s spouse also granted Statistics Canada permission to link to their tax records) will be used to match to the T1 file.

Using the SIN, a linkage will be made to T1 and T4 data for the each collection year (2007 through 2012) of the longitudinal pilot survey, and all previous calendar years going back to 1990. The employer pension plan identifier from the respondent’s T4 will be used to link information on the respondent’s employer pension plan from the Pension Plans in Canada File for each collection year only.

A separate linking key file containing the respondent’s SIN and other relevant personal identifiers will be retained for the life of LCS-Pilot processing (to the end of 2015).

Output: The data resulting from the linkage will be retained indefinitely by Statistics Canada’s Income Statistics Division. These data will have all personal identifiers removed and will be held with the rest of the content from the LCS-Pilot. All information released outside of Statistics Canada will conform to the confidentiality provisions of the Statistics Act. The linked income, tax, employment and pension data will be maintained in the LCS-Pilot database along with the respondent’s interview-obtained information. Other outputs from the project will include a summary of the pilot results, and other methodology-related reports.

---

Census of Agriculture: Feasibility Study of Linkage to Tax Data for Replacement of Farm Financial Questions

Purpose: This study will determine the feasibility of replacing respondent-provided farm financial data with tax data in the 2016 and future Censuses of Agriculture, with the goals of reducing response burden and improving the quality of farm financial data. For the study, Statistics Canada will link farm business tax data from the Statement of Farming Activities of T1 and T3 tax filers, and the income statement and balance sheet information for T2 filers, as well as the T4 Summary report, to the 2006 Census of Agriculture, the 2009 Census Test, and the 2011 Census of Agriculture.  The Farm Register will be used to facilitate tax and Census of Agriculture data linkage.

The results of this feasibility study will be used to develop procedures and systems for the proposed 2016 Census of Agriculture-tax data linkage. Aspects to be tested in the linkage include linkage methods, linkage keys, match rates, data quality, and willingness of farm operators to provide their Canada Revenue Agency (CRA) business number on the Census of Agriculture questionnaire for data linkage and data replacement purposes.

Description: Tax data on farm business revenue, expenses and payroll corresponding to farm financial questions on the Census of Agriculture will be obtained from Canada Revenue Agency files for farm businesses. The tax data will be evaluated for comparability to Census of Agriculture farm financial data collected from farm operators in the 2006 Census of Agriculture, the 2009 Census Test, and the 2011 Census of Agriculture and content test questionnaires.

Respondents to the 2009 Census Test and the 2011 Census of Agriculture will be requested to provide their farm CRA Business Number and will be notified of the linkage to tax data by means of a statement on the back of the Census form.

Output: Results of the research project will be used for evaluation, operational planning and quality assurance purposes. Outputs will include research or technical papers discussing the results of the research project. Linkage-based data from this study will not be disseminated as part of the initial 2011 Census of Agriculture database release.

If the study concludes that tax data replacement is not feasible for 2016, the linked files will be retained only until May 2017, at which time they will be destroyed. Should the study be successful, a decision will be taken at a later stage about whether tax data replacement will be implemented for 2016. At that time, a decision will also be taken as to whether the files that were linked for the feasibility study will be retained indefinitely, in order to furnish a data set that will be conceptually comparable to 2016 data.

---

Using 2006 Census of Population Data to Improve the Quality of the Estimates for the Access and Support to Education and Training Survey

Purpose: To improve the data quality of the Access and Support to Education and Training Survey (ASETS). ASETS is a new voluntary survey whose objective is to assess education and training demand in Canada in the context of life-long learning. The survey will provide information on access to post-secondary education (PSE), the role of student loans and savings in the financing of PSE, and participation in adult education and training. It will also provide information on special populations like aboriginal persons living off-reserve, recent immigrants and official language minorities. The data collected by ASETS will help to monitor preparedness and access to education, evaluate the effectiveness of government education-related programs and develop policies to deal with the training needs of Canadians. As such, it is important that the statistics generated from the survey be as accurate as possible.

The survey is based on a sample of 72,000 households. Despite extensive efforts made by the survey to maximize the response rate, residents in some dwellings could not be contacted and others chose not to participate in the survey. Thus, some nonresponse remains and the potential for bias in the survey estimates is significant, even if the differences between respondents and nonrespondents are small. The objective of the linkage is to use 2006 Census microdata as auxiliary information to improve the ASETS nonresponse adjustment methodology, reduce potential bias and ultimately improve the quality of the survey estimates.

Description: The survey selects its sample from a telephone list that was built using two sources: microdata from the 2006 Census of Population complemented by the list of telephone numbers used for Random Digit Dialling surveys. Sampled dwellings will be linked to the corresponding Census data to determine suitable classes to perform nonresponse adjustments for ASETS.

Output: Only aggregate information conforming to the confidentiality provisions of the Statistics Act will be released outside of Statistics Canada. Nonresponse classes and related variables will be kept internally, on the weighting file, which will be retained until December 31, 2011 at which time this file will be destroyed. The 2006 Census variables and nonresponse class identifiers will not be part of the ASETS master file, which will be kept indeterminately by Statistics Canada.

---

Evaluation of the Census Internet Response Option: Linking 2006 Census Data to the Evaluation Surveys Data

Objective: Allow Statistics Canada to offer Canadians an Internet response mode for the Census that will be secure and practical for respondents, and generate the highest quality data possible. This study will assist in establishing a promotion strategy for Internet response. In other words, establish a method to identify geographic areas where households will receive a letter requesting that they complete their questionnaire on-line without having received the questionnaire in advance.

The 2006 Census offered an Internet response option for the first time to all Canadian private households. It is expected that the participation rate (approximately 18% in 2006) will increase as households get connected and as respondents feel more at ease with this medium. With the 2011 Census it is expected that the Internet will probably be one of the main methods for collecting data.

In Canada, the quinquennial census is the sole source of reliable detailed data for small areas such as neighbourhoods and for small groups such as single parent families, immigrants, as well as industrial and professional categories. All levels of government, as well as enterprises, industries, the media, academia and independent organizations benefit from census data which are essential to their decisions regarding many public services.

Federal transfer payments that are calculated in establishing various provincial and territorial programs are based on census population counts. Several provincial and regional governments also use population counts to grant subsidies to local and municipal governments.

Governments use census data extensively in developing their plans and policies in sectors such as employment, schools, training, transportation, housing, immigration and income support. Urban planners and community infrastructure planners rely on census data to plan for schools, roads, water supply systems, public transport, fire protection, and future requirements for housing, hospitals and daycare.

Description: The 2006 Census data will be matched with the data collected through three evaluation surveys that will assess the 2006 Internet response. These surveys were conducted on a voluntary basis among 5200 respondents to the 2006 Census. The respondents were informed that to reduce the number of questions asked, information obtained during the Census would be added to the evaluation surveys data for analysis purposes. The data will be matched using a household identification number.

Product: The aggregate data will be presented in a Statistics Canada internal report. The linked files will be kept until May 31, 2011, and then destroyed.

---

Longitudinal Immigration Database and Immigrant Information in the Longitudinal Administrative Databank: 2005 to 2009 Updates

Purpose: To provide information on the economic integration of immigrants into Canadian society. The Longitudinal Immigration Database, known as IMDB, enables the federal and provincial governments involved in immigration issues and programs, as well as the research community and immigrant settlement agencies in Canada, to conduct research regarding the selection of immigrants, their settlement patterns and their economic integration into Canadian society. The IMDB is the only source of data that can support research on the impact of immigration policy levers such as the category of admission, selection criteria and special admissions program, on economic outcomes, and is the only source of longitudinal data with a sufficient sample of immigrants to examine settlement trajectories and integration patterns over time and by characteristics at arrival and selection criteria.

Since the Census of Population does not collect information on the human capital of immigrants upon their arrival to Canada or policy levers in effect at that time, the IMDB is unique in addressing specifically the impact of admission categories and foreign credentials on economic integration.

The addition of data for the 2005 to 2009 period in the IMDB will continue to enhance the database in order to inform immigration policy and programs.  In particular, the information will allow the analysis of the economic outcomes of immigrants who came to Canada during the 1990s and 2000s, as well as the evaluation of immigrant selection under the 2002 Immigration and Refugee Protection Act.

The IMDB is supported by a federal-provincial consortium, led by Citizenship and Immigration Canada, that includes Human Resources and Social Development Canada and the provincial governments of Quebec, Ontario, Manitoba, Saskatchewan, Alberta and British Columbia. These federal departments and provincial governments have worked together for a number of years to direct the development of the IMDB and to support the development and maintenance of the database at Statistics Canada.

The linkage and analysis will be conducted in Statistics Canada’s offices. The linked files will not be available outside Statistics Canada.

Description: The IMDB links immigration and immigrant landing administrative files with information from tax files of immigrants who are granted the right to live permanently in Canada. Currently, the database includes records of immigrants who arrived in Canada from 1980 to 2004. Information for those who arrived in Canada from 2005 to 2009 and new tax data on those immigrants already in the database will be added as the data become available. Once an individual immigrant’s data have been successfully matched for the first time, the record for this person is retained in the IMDB and continues to be matched annually to tax files for up to 16 years. Once the linkage is done, all names and Social Insurance Numbers are removed from the analysis files and stored separately on password-protected files. Also, immigration identifier and landing information are provided annually to the Longitudinal Administrative Databank (LAD) program of Statistics Canada.

Output:  Only aggregate data conforming to the confidentiality provisions of the Statistics Act are released outside of Statistics Canada. The IMDB will be retained until the next review, planned for 2011, to ensure its continued relevancy to immigration policy development. The immigration identifier and landing information are retained indefinitely on the LAD.  Both the IMDB and the LAD are registered with the Treasury Board of Canada as Personal Information Banks, namely PPU 135 and PPU 112, respectively.

---

Omnibus Record Linkage Authority for Improving the Population and Household Survey Programs

Purpose: To assess and improve the quality of statistical outputs from Statistics Canada’s population and household survey programs; to reduce survey costs and reduce response burden for household surveys.

Statistics Canada carries out record linkage activities as part of its population and household survey programs. Linkages covered by this omnibus authority have one of three purposes:

1. To obtain information that benefits a survey, such as for stratification in survey design, but that does not directly contribute to estimates;
2. To study and assess survey data quality, for example, by comparing survey data to data from other sources; and
3. To aid in data collection, such as to provide addresses to mail introductory letters or to provide telephone numbers to reduce collection costs by permitting data collection through a telephone interview.

These types of linkages do not contribute directly to statistical outputs that are disseminated outside Statistics Canada. Linkages that do contribute directly to statistical outputs for public release are not covered here and require separate approval.

Description: The linkages can be survey file to survey file, survey file to administrative file, or administrative file to administrative file. The linkages covered by this authority do not introduce a disclosure risk since they do not involve activities to directly produce statistical outputs.

Output: Linkages will not result in statistical outputs for dissemination outside Statistics Canada. For a given survey, no data from sources outside the survey, obtained from a linkage, will be used as individual values for estimation.

Linked files will be held solely within Statistics Canada, and will be retained only for the duration of the project (at most a few years). As of November 1, 2008, Statistics Canada will maintain an inventory of all linkages conducted under this omnibus authority.

Audit Report

April 22, 2013
Project Number: 80590-76

• Executive summary
• Introduction
  • Background
  • Audit objectives
  • Scope
  • Approach and methodology
  • Authority
• Findings, recommendations and management responses
  • Long Term Planning process control environment
  • Oversight and scrutiny for decision-making
• Appendices
  • Appendix A: Audit criteria
  • Appendix B: Acronyms
• Figures
  • Figure 1 - Integrated Strategic Planning Process (ISPP)
  • Figure 2 - Stages of the Long-Term Planning Process

Executive summary

With the mandate to serve the statistical needs of all levels of government, Canadian institutions and all Canadians, Statistics Canada seeks to ensure that Canadians have access to a trusted source of statistics on Canada that meet their highest priority information needs. The provision of high quality data supporting core economic, environmental, and social statistics programs is a key driver in determining how Statistics Canada allocates its funding. As such, strategic priority setting and funding of investments is achieved through the Integrated Strategic Planning Process (ISPP) established in 2010-11. The ISPP is an annual six step cycle, the first four steps of which are known as the Long Term Planning (LTP) process established in 1984.

Statistics Canada plans to invest $134 million in assets, $171 million in acquired services and $512 million in projects (largest project is the 2011 Census of Population and National Household Survey) over the 2011-12 to 2015-16 planning horizon. The planned investments meet the Agency's operational requirements and are within the Agency's existing reference levels.^{Footnote 1}

The objectives of this audit were to provide assurance to the Chief Statistician (CS) and Statistics Canada's Departmental Audit Committee (DAC) that governance mechanisms are in place to support the planning, resource allocation and monitoring of LTP projects, and the identification and approval of LTP projects is enabled by a structured and systematic process.

The audit was conducted by Internal Audit Services in accordance with the Government of Canada's Policy on Internal Audit.

Key findings

The Agency has a formal LTP process that is documented and generally understood by participants. However, some important roles and responsibilities are not well defined, leading to a lack of clarity in relation to how key stakeholders are involved in the process. In other instances, roles and responsibilities of key governance bodies are not well defined, or defined with a high degree of flexibility, creating opportunities for inconsistency in governance over the planning process.

There is currently no requirement to formally document planning assumptions in support of costing and efficiencies estimates; and formal guidance for the preparation of the estimation of costs, planned and actual efficiencies is not provided. As a result, the approaches used to generate and track these estimates are inconsistent.

Currently, Corporate Business Architecture (CBA) projects do not always take into consideration the interdependencies that exist within Statistics Canada's matrix environment. This may result in the loss of opportunities for greater efficiencies. An 'Interdependency Model' has been developed by Corporate Planning Division (CPLD) to better identify and manage interdependencies. Approval of the model will make it mandatory for CBA projects to formally track, identify, manage and monitor interdependencies amongst CBA projects.

Governance and operational practices for scrutiny and challenge vary across Fields, with limited substantive review and challenge of the financial and non-financial information in the LTP forms by PAs and FMAs. Challenges by key participants and within governance bodies are not documented. This may result in inconsistent assessment of proposals across Fields and incomplete information to support defensible planning and approval decisions.

There is general compliance with the LTP process as mapped and value of the LTP process is increasingly being realized with the considerable evolution in the use of the DPMF and related tools. However, original baseline costs and efficiencies estimates of projects should be included in the Executive dashboard to systematically track, monitor, understand and challenge the impact of project adjustments from original cost and efficiency estimates.

Record-keeping and information management practices related to ownership, version control and transmission should be strengthened to ensure sound stewardship and data integrity.

Overall conclusion

The LTP process has been established for some time, but has recently seen a major evolution with the advent of the DPMF and related requirements. The new structures and tools for the LTP process provide a good foundation for discipline and defensible decision-making, but more progress is needed – both process wise and culturally.

Greater clarity on roles and responsibilities; guidance on cost and efficiency estimation; and formal documentation of planning assumptions for costing and efficiency estimates will strengthen the ability of governance bodies and key participants to effectively discharge their scrutiny and challenge responsibilities. Inclusion of the original baseline costs in the project dashboards will strengthen the ability of project sponsors and governance bodies to effectively track and monitor the cumulative impacts of project adjustments to ensure sound stewardship of resources.

Opportunities exist to strengthen record-keeping and information management practices related to ownership, version control and transmission to ensure sound data stewardship and data integrity.

Conformance with professional standards

The audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada, which includes the Institute of Internal Auditors (IIA) Internal Standards for the Professional Practice of Internal Auditing.

Patrice Prud'homme
Chief Audit Executive

Introduction

Background

As Canada's national statistical office, Statistics Canada has a mandate to serve the statistical needs of all levels of government, Canadian institutions and all Canadians. Statistics Canada contributes to most federal government outcomes in the domains of economic, social, international and government affairs. In so doing, it seeks to ensure that Canadians have access to a trusted source of statistics on Canada that meet their highest priority information needs.

The allocation of funding in support of the Agency's mandate and priorities is done through the Integrated Strategic Planning Process (ISPP) established in 2010-11.The ISPP describes how decisions are made through the long term planning (LTP) process. The ISSP is an annual process made up of six steps. The first four steps are the LTP process and the fifth and sixth steps relate to the communication of plans and the monitoring of LTP projects. Established formally in 1984, the LTP process has evolved over the years, most recently adapting to the application of the tools implemented as part of the Departmental Project Management Framework (DPMF) which itself was launched as a pilot in 2011-12. Project management functions are supported by the Departmental Project Management Office (DPMO).

Through the LTP process, the Agency annually assesses its programs and resources to adjust to changing client needs and priorities. It also provides a forum for the identification and discussion of cross-cutting issues, fostering efficiency improvements and encouraging innovation. To ensure ongoing program continuity and quality, the ISPP exercise features a ten-year planning horizon called "Funding the Future" to help the Agency plan significant expenditures well in advance, manage operational risks and align its programs to the needs of Canadians. Through the ISPP, the Agency integrates sound management practices such as risk management, investment planning, project management and evaluation into the planning process.

Figure 1 - Integrated Strategic Planning Process (ISPP)

Description for Figure 1 - Integrated Strategic Planning Process (ISPP)

The first step of the LTP process starts in April in each year with a "strategic direction" session to align strategic direction with priorities and emerging issues, update the Corporate Risk Profile and focus the discussions around mitigating important corporate risks. At the second step from May to June, managers update their 10 Year Investment Outlook to reflect the most current information available, and develop a high level business proposal to request funding through the LTP process. Proposals are grouped into three main categories for decision-making:

1. Corporate Business Architecture (CBA) improvements and initiatives;
2. Continuity and Quality maintenance (CQM) of existing programs; and,
3. New Initiatives / Enhancements.

Each category reflects the different aspects of strategic planning. CBA initiatives find new and better ways to do business and either generate efficiencies in corporate business processes or improve their robustness and responsiveness. Net efficiencies generated by CBA projects are used to maintain the continuity and quality of existing programs through the CQM projects, such as, implementation of new international standards and classifications, system redesigns, sample redesign, etc. New initiatives or program enhancements are funded by re-allocating existing budget or new monies (Treasury Board allocations or cost-recovery funds).

During the third step, from July to October, a detailed business case for each approved proposal is developed by the managers. Full costs, options for investments and anticipated savings are identified for review and approval by the Field Planning Boards (FPBs).

At the fourth step in November, business cases are presented to the Senior Management Review Committee (SMRC). SMRC ensures that all information and implications have been duly considered, and determine which investments will be funded. The final decisions are recorded in the Decision Record (Blue book) which is distributed to the Agency's senior management team at the conclusion of the process.

Audit objectives

The objectives of this audit were to provide assurance to the Chief Statistician (CS) and Statistics Canada's Departmental Audit Committee (DAC) that:

• The Agency has appropriate governance mechanisms in support of the planning, resource allocation and monitoring of LTP projects; and
• The identification and approval of LTP projects is enabled by a structured and systematic process, supported by appropriate knowledge and defensible information inputs.

Scope

The scope of this audit included an examination of the adequacy and effectiveness of the controls in place over the Long-Term Planning process. Specific areas that were examined included:

• Governance and monitoring practices that support clear accountability, oversight, scrutiny and challenge functions; and
• Operational processes and controls that enable the consistent application of a common LTP methodology as well as the tools, training and information management practices that support informed LTP decisions.

The scope of the audit included all Fields of the Agency and examined a sample of 20 project files approved in 2010-11, for 2011-12 LTP funding. The audit acknowledges that the fiscal period selected for testing was the first year of launching the DPMF as a pilot and the programs and the DPMO were adapting to the new process.

Approach and methodology

The audit work consisted of examination of documents, interviews with key Senior Management and personnel, and a review of relevant procedures, guidelines and frameworks related to the Long Term Planning process.

The field work consisted of a review of applicable procedures and documentation to support the governance and monitoring practices; and testing of operational processes and procedures to verify if appropriate and robust information management practices are in place; if the LTP process is adhered to by all key stakeholders; and if robust scrutiny and challenge mechanisms exist and are applied in support of defensible planning and approval decisions.

This audit was conducted following the Standards for the Professional Practice of Internal Auditing of the Institute of Internal Auditors (IIA) and in accordance with the TBS Policy on Internal Audit.

Authority

The audit was conducted under the authority of Statistics Canada Integrated Risk-Based Audit and Evaluation Plan 2012/13-2014/15 recommended by the Departmental Audit Committee, April 2012 and subsequently approved by the Chief Statistician.

Findings, recommendations and management responses

Objective # 1:  The Agency has appropriate governance mechanisms in support of the planning, resource allocation and monitoring of LTP projects.

Long Term Planning process control environment

Treasury Board Policy on Investment Planning – Assets and Acquired Services requires Deputy heads to ensure that appropriate governance, systems and resources for investment planning are in place, and maintained based on principles of sound stewardship and value for money. As well, an effective investment management framework, including departmental procedures, processes and systems should be implemented.

Long Term Planning process

A description of the Long Term Planning process is embedded in two key documents, "Integrated Strategic Planning Process at Statistics Canada: An Overview" and the "Departmental Project Management Framework (DPMF) Guidelines" (the Guidelines) developed by the Corporate Planning Division (CPLD).

The Integrated Strategic Planning Process at Statistics Canada: An Overview provides an explanation of the ISPP; a description of the governance structure specific to the ISPP with a description of the key responsibilities of its governance committees. It includes a separate section for each of the six steps, with a description of the specific activities, key stakeholders, responsibilities and the documents, deliverables and outputs related to each activity.

The LTP process consists of two stages within the project management framework, and the Guidelines provide a description of project management concepts; project risk management; project management stages, deliverables and documentation; the governance process, governance committees and description of their roles and responsibilities; and a description of the roles and responsibilities of the project manager and project teams.

Both documents are easily accessible to all key process participants on Statistics Canada's intranet site. Training on the ISPP is in place and has been provided on a pilot basis. The training material provides a good overview and description of the LTP process and is consistent with the Guidelines.

Figure 2 - Stages of the LTP Process

The two stages of the LTP process are illustrated below.

In Stage 1 programs develop a Business Proposal (BP) for approval. Corporate Business Architecture (CBA) proposals go through an approval process by both the CBA management committee and FPBs and non-CBA projects go through an approval process by the FPBs. Proposals approved by the FPBs are then presented to the SMRB in June, for gate 1 approval and consideration for LTP funding. Proposals approved for consideration in June, move to Stage 2. In Stage 2 programs prepare a Project Complexity and Risk Assessment (PCRA), Business Case and Business Case Costing (BCC) template. These forms then go through the same approval process as in Stage 1 and are presented to the SMRB in November for gate 2 approval.

Roles and responsibilities

Roles, responsibilities and accountabilities for the following governance committees involved in the LTP process, are formally defined and documented in the ISPP document and the Guidelines:

• Field Planning Boards (FPBs)
• Corporate Business Architecture (CBA) management committee
• Corporate Business Architecture Secretariat
• Senior Management Review Board (SMRB)
• Branch Steering Committee and
• Project Steering Committee

The role of the FPB is defined in general terms in the Guidelines. The Guidelines stipulate that

"each FPB has the flexibility on how they conduct their gate reviews".

This introduces a measure of inconsistency into the planning process, including the timeliness of DPMF document submissions for the LTP process. This could negatively affect the consistency of governance practices across Fields.

The audit noted that formal Terms of Reference do not exist for any of the oversight bodies, although, one of the Fields has developed a 'Governance Model' document which describes the purpose of the FPB and other branch level oversight bodies and their roles and responsibilities with respect to the LTP process. Terms of Reference are written guidelines that clarify the role, purpose and responsibilities given to a committee. SMRB is ultimately responsible for the decisions taken by the governance bodies; therefore it is important to establish a clear reporting mechanism and to set the perimeters within which authority is delegated to the FPBs and the CBA management committee.

Interviews with the FPMs for the other fields revealed their intention to replicate the 'Governance Model' for their own Fields. The CBA secretariat also confirmed its intention to develop a formal Terms of Reference for the CBA management committee.

Roles, responsibilities and accountabilities for the Project Manager (PM) and the Project Team are formally defined and documented in the Guidelines. This is not the case for the Planning Analyst (PA), Field Portfolio Manager (FPM) and the Financial Management Advisor (FMA) which has resulted in some confusion and overlap of activities.

The role and responsibilities of the PM varies across Fields and programs with respect to the LTP process. PMs at assistant director level and above were more engaged in the process from the idea identification and generation stage and participated at branch and divisional management committees and at the FPBs. The role of the PMs at the Chief level generally started with the preparation of the DPMF documents for stage 2 of the LTP process and they do not benefit from the knowledge of the complete life-cycle of the proposal.

The Planning Analyst is understood to be the owner of the LTP process but a common understanding of the nature of the scrutiny they should be exercising is not defined. As owners of the LTP process, they exercise their process knowledge, by ensuring that documents presented before the FPBs align with corporate objectives and the DPMF requirements, with a focus on their completeness and reasonableness, but not on their substance or content of the information submitted. Interviews with Project Sponsors (DG level) confirmed that PAs assist in data entry and ensure comparability of proposal submissions, and that a common approach is followed for all projects, but do not perform a challenge function.

The position of the FPMs who report to Assistant Chief Statisticians (ACS) is new and has emerged due to the increase in oversight and documentation requirements as part of the uptake of the DPMF. The FPMs assist with the management and monitoring of project activities within their Field. As such, they review the DPMF documentation for completeness before submitting to the FPB and submit the final version to the DPMO. There is overlap and duplication in the roles and responsibilities of the PA and FPM.

The role of the FMA, like the PA, varies across Fields. For instance, in some Fields, they are present at the Field Planning Board on a mandatory basis. In others, they are invited on an as-and-when-required basis. Their roles and responsibilities with regards to scrutinizing and challenging financial estimates during the LTP process are not clear.

A formal LTP process is documented and generally understood by participants, but some key roles and responsibilities are not well defined, leading to a lack of clarity in relation to how key parties are involved and contribute to the process.

Guidance

The LTP process is embedded in the broader guidance around the ISPP and the DPMF, and is described at a high level, stage-by-stage in the portion of the DPMF mechanisms in the Guidelines related to the LTP process. Direction for each stage focuses on the forms to be prepared and describes who is responsible, accountable, consulted and informed for each deliverable.

Interviews identified that there was insufficient guidance on the preparation of cost estimates and the estimation of planned efficiencies and actual efficiencies realized (for CBA projects). Review of the Guidelines confirmed this to be the case and revealed that assumptions are not required to be documented as supporting explanation for how the costs or efficiencies estimates presented were arrived at. Interview evidence suggests that discussions on planning assumptions take place informally and bilaterally at various stages of the LTP process. The discussions are based on the accumulated and combined experience and knowledge of the key parties engaged in the process, however, little to no documented evidence exists of these discussions.

Review of the DPMF documents for the LTP process (the Business Proposal, the Business Case and the Business Case Costing templates), revealed that each form template is supported by a guide, which provide detailed instructions on how to prepare each form. In the guide for the Business Proposal template, the definition that is included for an assumption is

"that which would make the proposal successful or unsuccessful".

It does not, however, specify that the planning assumptions for costs or efficiencies estimates should be included.

At the time of the audit, CPLD indicated that they will be implementing a 'Project Costing' template, which would require the documentation of financial and non-financial inputs, and underlying assumptions. This will reduce the risk of an inconsistent and incomparable approach to costing and efficiencies estimates and increase the ability to track the achievement of efficiencies.

As a matrix organization, functional responsibilities at Statistics Canada are organized into six Fields, with the management of product development processes for large projects taking place across the Fields. As a result, most projects are inter-dependent in nature, and require consultation across the organization to identify potential synergies, allow for appropriate linkages, and manage changes across Fields. Projects not managed in an inter-dependent fashion could result in overlap, duplication and unnecessary costs.

The audit found that when resource estimates are being prepared, the Guidelines do not outline any specific requirements for information gathering or consultation across Fields or with internal service providers. In practice, some cross-Field consultation takes place informally or bilaterally at the project level, but there is no documented evidence to suggest that this is done. As a result, limited audit trail is in place regarding the nature and extent of deliberations and discussions held, and of any assumptions made. Sign-off from inter-dependent Fields was a requirement as part of the oversight processes in the past, but with the DPMF, forms are now submitted electronically and physical sign-offs by partners are not required.

An 'Interdependency Model' has been developed by CPLD to better identify and manage interdependencies. Once the model is approved, it will be mandatory for CBA projects to formally track, identify, manage and monitor interdependencies amongst CBA projects.

Recommendations:

The Assistant Chief Statistician (ACS) Corporate Services should ensure that:

• The roles and responsibilities of the Planning Analyst, the Field Portfolio Manager, the Financial Management Advisor, the Program Manager and the Field Planning Boards are clearly defined, documented and communicated in the Guidelines; and formal Terms of Reference for the governance bodies (Senior Management Review Board, CBA management committee ands the FPBs) are developed, approved and reviewed periodically to establish role, purpose and responsibilities and include clearly identified timelines between stakeholders for delivery of key documents for the LTP process.
• The DPMF LTP process templates (Business Proposal, Business Case and Business Case Costing) are updated to require that planning assumptions and rationales for the cost and efficiency estimates (where applicable) be documented; and a Project Costing template is developed and adopted for a consistent and comparable approach to cost and efficiency estimation.
• Guidance is developed on the conduct and documentation of interdependency consultations between Fields, including internal service providers, to formally track, identify, manage and monitor interdependencies.

Management response:

Management agrees with the recommendations.

• The Director CPLD will ensure the documentation of the roles and responsibilities of the Planning Analyst, the Field Portfolio Manager, the Financial Management Advisor, the Program Manager and the Field Planning Boards and this is in progress.
  
  Deliverable and Timeline: The "Field Planning Board Roles and Responsibilities: Terms of Reference" document will be made available to all managers on CPLD ICN site, by June 30, 2013.
• The Director CPLD will ensure the documentation of Formal Terms of Reference for the governance bodies (Senior Management Review Board, CBA management committee ands the FPBs) with clearly identified timelines for delivery of key documents for the LTP process will be documented.
  
  Deliverable and Timeline: Formal Terms of Reference will be made available to all managers on CPLD ICN site, by October 31, 2013.
• The Director CPLD launched a process and tool to monitor and control CBA Portfolio risks, issues and interdependencies for CBA projects and 16 non-CBA projects as of Feb 28, 2013.
  
  Deliverable and Timeline: Completed interdependencies spreadsheet and analysis of project key deliverables and timelines, by June 30, 2013. Training and support to project managers, by August 31, 2013 and Quarterly updates to CBA management committee by October 31, 2013.

Objective #2:  The identification and approval of LTP projects is enabled by a structured and systematic process, supported by appropriate knowledge and defensible informational inputs.

Oversight and scrutiny for decision-making

Appropriate and robust stewardship practices, scrutiny and challenge mechanisms should exist to support defensible LTP decisions; provide stewardship of project resources; create accountability for projects and align projects with departmental strategic outcomes.

The audit tested a sample of 20 LTP project files approved in 2010-11, for 2011-12 LTP funding to verify if appropriate and robust information management practices are in place; if the LTP process is adhered to by all key stakeholders; and if robust scrutiny and challenge mechanisms exist and are applied in support of defensible planning and approval decisions. The audit acknowledges that the fiscal period selected for testing was the first year of launching the DPMF as a pilot, and notes that both the programs and the DPMO were adapting to the new process.

Seven CBA and thirteen non-CBA projects were judgementally selected for testing to ensure coverage of the three categories of projects from all the Fields. Completed Business proposal (BP), Business Case and Business Case Costing (BCC) forms for each project were requested from the Corporate Planning Division (CPLD) which is

"the central repository of all electronic copies of final documents"

per the Guidelines.

Information management

Review of the Guidelines revealed that although document management is discussed, the focus is largely on the templates to be used, not the information management practices related to ownership, version control, controlled transmission and data integrity. The audit sought to trace the sampled BP, Business Case and BCC forms held in the central repository and confirm that they were in agreement with those records held by the project manager. This test was not completed because the audit team encountered difficulty in obtaining the final documents. For the CBA projects, CPLD was only able to provide two thirds of the documents requested and for non-CBA projects they were only able to provide half of the documents requested. A common repository for non-CBA projects was not required for the year selected for our audit test, but has since been established in the subsequent year.

A review of the structure of the DPMO repository revealed that it is configured to follow the stages of the ISPP and all the LTP project folders were consistently mapped to this structure. As noted above the project folders for CBA projects did not contain the complete project documentation. Further enquiry revealed that record keeping challenges were attributed to PA turnover and the volume of the projects to manage. The audit noted that with the advent of the FPM, the role of the PA as coordinator and manager of information is not clear and may result in confusion over version control and ownership of the LTP documents.

Adherence to the LTP process

• The BP forms for 6 CBA projects and 9 non-CBA projects received were completed in compliance with the instructions in the guidance included in the form.
• Only 3 CBA projects and 6 non-CBA projects had developed a Business Case. For the CBA projects, 1 of the forms had been well completed with assumptions provided, while those for the other 2 appeared to be under development, with many sections uncompleted.
• Of the projects sampled, 2 did not have Business Case Costing forms. The remainder contained the estimated costs and planned efficiencies.

For projects that had completed forms on file, the audit noted that little to no supporting documentation of assumptions or supporting analysis was available for the cost and efficiency estimates.

The audit noted that there has been a considerable evolution in the use of the DPMF and related tools, particularly with respect to CBA projects and the value of the process are increasingly being realized. For non-CBA projects, the LTP process is still being viewed as a 'compliance' requirement rather than a valuable management practice. However, record-keeping and information management practices related to ownership, version control and transmission should be strengthened to ensure sound stewardship and data integrity.

Scrutiny and challenge mechanisms

Scrutiny and challenge of an LTP project proposal from idea generation to gate 1 and gate 2 approvals takes place at the Field level and at the departmental level. Given the volume of project proposals that are identified for LTP funding, much of the scrutiny and challenge function takes place in advance of the departmental level governance meetings. This is appropriate both from a process flow and from an efficiency perspective.

At the Field level, interview evidence revealed that a considerable amount of bilateral verbal discussions occur in varying ways by key participants in the scrutiny and challenge of project proposals and the LTP documents. Rigour and due diligence is performed by senior management (Assistant Directors, Directors, DGs, ACSs), at various branch and divisional meetings. Project sponsors, i.e. DGs are involved from the idea generation stage and see all the projects as part of the early decision making within their areas. According to them, assumptions are not documented in the DPMF forms, but are discussed verbally and challenged at the various Field level meetings; Project Steering Committee meetings; and finally the FPB. The nature of the scrutiny exercised by the key participants could not be independently examined due to lack of documentary evidence.

The manner in which PAs, FPMs and FMAs within a Field interact with each other and management in the substantive scrutiny and challenge of project proposals and LTP documents varies, and is based on the individual Field and personality of players.

Interviews revealed that PAs do not always feel they have the authority to exercise due diligence or the time for a substantive review and challenge of the financial and non-financial information in the LTP forms. FMAs involvement in the LTP process starts mainly at project implementation. Like the PAs, they also feel they do not always have the authority to exercise due diligence or the time for a substantive review and challenge of the financial information in the LTP forms. FPMs are increasingly playing a coordination and scrutiny role similar to that performed by the PAs.

At the Field level, the FPBs are in place and play an active role in the scrutiny and challenge function of the LTP project proposals. However, the Guidelines stipulate that

"each FPB has the flexibility on how they conduct their gate reviews".

This introduces a measure of inconsistency into the planning process which could result in variable governance practices across Fields. Review of FPB meeting presentations and minutes revealed that their practices do vary across Fields.

At the departmental level, CBA management committee and the SMRB are in place. Interview evidence, review of CBA management meeting presentations, and attendance as observers by the audit team at a CBA management committee and the SMRB conference in November 2012, revealed that bilateral verbal discussions and deliberations, where feedback is sought and provided, take place. Information, knowledge, and expertise are exchanged and assumptions on cost and efficiency estimates are covered, challenged and discussed and approval is sought, given, and the decisions are documented in the records of decisions or minutes of meetings. The audit concluded that the CBA management committee plays an active and robust role in scrutinizing and challenging CBA LTP projects.

The audit reviewed the minutes of the FPBs and the CBA management committees charged with approval, for the projects in our sample to verify if gating and key decisions were formally approved for LTP funding. The audit noted that the CBA management committee reviewed and formally approved all of the CBA projects at stage 1 and stage 2. The FPBs reviewed and discussed the projects over several sessions before formally approving them and submitting them to the SMRB for gate 1 and 2 approval.

Governance and operational practices for scrutiny and challenge vary across Fields, with limited substantive review and challenge of the financial and non-financial information in the LTP forms by PAs and FMAs. Challenges by key participants and within governance bodies are not documented. This may result in inconsistent assessment of proposals across Fields and incomplete information to support defensible planning and approval decisions.

Monitoring: Post-approval feedback loop for variances and adjustments

The audit examined, in Stage 4 of Project Implementation, the nature of the monitoring controls that allowed the governance bodies, charged with approval, to understand and oversee subsequent adjustments to key project variables. This did not constitute a full examination of project monitoring controls, but was based on targeted testing of the feedback loop that informs governance bodies of major variances from the original commitments. This was done because there is inherent risk that project proposals are optimistically presented for the purposes of project approval, only to have the cost estimates adjusted upwards, or the efficiency adjusted downwards, post-approval.

The audit found that once the project is in production, monitoring of project variables is undertaken through a variety of mechanisms.

For non-CBA projects, responsibility for project monitoring rests with the project steering committee and other governance bodies established for a particular project. Adjustments to the original baselines (cost, timelines, etc), are required to come before the FPB for review and approval. This is done through a formal adjustment process, whereby the adjustment is put forward through the LTP process as a new proposal and follows the overall LTP process (i.e., uses the same templates and governance structures, etc.). Since the assumptions underpinning the original cost estimates are not formally documented, the FPB relies on the corporate memory of those involved. This is a compensating control, but it is a weak control as it relies on the permanency and availability of the key participants and could therefore limit the degree and ability of the FPBs to effectively challenge and scrutinize the changes. In April 2012, ten top non-CBA projects were transitioned by the DPM office to use the Executive Dashboard for review and monitoring by the CBA management committee.

For CBA projects, steering committees and FPBs also exercise monitoring responsibilities. The CBA management committee also exercises review responsibilities, using the Executive Dashboards, Executive Portfolio Summary and periodic risk reporting as their main tools. The Dashboards provide in-year reporting, fed both by FRS and by the project managers themselves. In-year variances in the cost or projected efficiencies are tracked, but no link to the original baseline data is presented or tracked. Therefore, the health of the project is compared to the adjusted estimates of cost and efficiencies and not to the original baseline costs. In examining the degree of scrutiny exercised by the CBA management committee over adjustments, the audit reviewed minutes and found that there was thorough scrutiny of both non-financial and financial factors. Nonetheless, because the original baseline costs and efficiencies estimates are not noted in the project dashboard, the ability of governance bodies to systematically track, monitor, understand and challenge the changes is limited and weakens the quality of information required for defensible sound decision-making.

Corrective action and feedback loop appears to be for tracking milestones and not for measuring if funding requested was sufficient and if efficiency estimates are realistic. There is also a risk that efficiencies may be "harvested" from a program area on the assumption that efficiencies have been realized, but due to the inability to effectively measure them, they have not. This could result in an unfair "taxing" of the organization which could have negative consequences in terms of program integrity.

Recommendations:

The Assistant Chief Statistician (ACS) Corporate Services should ensure that:

• Record keeping practices are strengthened with regards to version control, access to key planning information (i.e. assumptions, approvals, corrective action plans, etc.).
• The Executive Dashboard is enhanced to include original baseline costs and efficiencies estimates of projects, to allow for the monitoring of cumulative impacts of project adjustments from original costs and efficiency estimates.

Management response:

Management agrees with the recommendations.

• Director CPLD, is currently conducting a review of the revised structure of the DPMO repository, and on an annual basis, will conduct a formal review to ensure the DPMO repository is up-to-date.
  
  Deliverable and Timeline: Documentation of the revised DPMO repository structure by October 31, 2013.
• The Director CPLD, will perform regular monitoring to ensure the correct versions documents have been stored in the DPMO repository.
  
  Deliverable and Timeline: Ongoing monitoring of the correct versions documents in the DPMO repository.
• The use of the Executive Dashboard and baselining of project costs come into effect at DPMF Stage 4 (Project Planning). The Director CPLD, will ensure that the Executive Dashboard is enhanced to incorporate the changes recommended during the major review and update to the Executive Dashboard planned as part of the CQMIP for Corporate Planning Division in 2014-15. Additional requirements and enhancements to the dashboard, including original baseline costs and efficiencies estimates of projects, will be gathered and incorporated accordingly.
  
  Deliverable and Timeline: Updated Executive Dashboard by October 31, 2014.

Appendices

Appendix A: Audit criteria

Appendix A: Table 1 Audit criteria
Table summary
The table in Appendix A identifies the Audit Criteria, audit sub-criteria as well as the policy instrument used as the source of these criteria.

Objectives /Core Controls / Criteria	Sub-Criteria	Policy Instrument
1) The Agency has appropriate governance mechanisms in support of the planning, resource allocation and monitoring of LTP projects.
THEME: Governance
1.1 Authority, responsibility and accountability in relation to the planning, oversight and monitoring of LTP projects are clear, communicated and is applied as intended.	1.1.1 Authority, responsibility and accountability of oversight bodies and key individuals involved in the LTP process are clearly defined and communicated.	TBS Core Management Controls Management Accountability Framework TBS Policy on the Management of Projects TBS Policy on Investment Planning Statistics Canada's "Departmental Investment Plan 2011-12 to 2015-16" Statistics Canada's "Integrated Strategic Planning Process at Statistics Canada: An Overview" Statistics Canada's "Departmental Project Management Framework Guidelines"
	1.1.2 Authority, responsibility and accountability are exercised as intended.
1.2 A formal, timely constructive and substantive challenge function exists in relation to planning inputs (financial and non-financial information) prior to it being reviewed by the governance bodies.	1.2.1 There is guidance and documentation on the process steps and key accountabilities.
	1.2.2 There is challenge and scrutiny of the planning inputs (financial and non-financial information) at all levels (e.g. Project, Field, Branch, Agency levels).
	1.2.3 There is guidance and feedback to programs on identified costs and planned efficiencies.
1.3 The oversight bodies at the Agency, Field, Branch and Project levels receive sufficient (i.e., complete and comprehensive), timely and substantiated information in support of their planning and approval decisions.	1.3.1 Information to support oversight bodies in review and approval decisions are defined.
	1.3.2 Oversight bodies receive sufficient, substantiated and timely information in support of their planning and approval decisions.
1.4 LTP projects are formally approved and endorsed by the appropriate governance structures.	1.4.1 Gating and key decisions are formally endorsed and noted as approvals.
1.5 Financial and non-financial data used for project monitoring purposes is scrutinized and is reliable, defensible and meaningful.	1.5.1 There is monitoring by governance bodies during the project execution stage.
	1.5.2 Data and information presented in the LTP forms has integrity.
1.6 Corrective action is taken and monitored in a meaningful and systematic fashion when material variances in project performance are noted. Management is held accountable for implementing corrective action.	1.6.1 There is an established and understood process for managing material variances in project variables.
	1.6.2 Corrective action plans are put in place when variances are noted through dashboards.
2)  The identification and approval of LTP projects is enabled by a structured and systematic process, supported by appropriate knowledge and defensible informational inputs.
THEME: Stewardship
2.1 The Agency has a common and well-defined process for long-term planning that is adhered to by all relevant stakeholders.	2.1.1 A common and well-understood process exists.	Statistics Canada's "Integrated Strategic Planning Process at Statistics Canada: An Overview" Statistics Canada's "Departmental Project Management Framework Guidelines"
	2.1.2 The LTP process is adhered to.
2.2 LTP stakeholders are provided with the necessary training, tools, resources and information in support of their responsibilities.	2.2.1 Guidance and training material exists and are provided to LTP participants.
	2.2.2 Common guidance and methods exists on costing and identification / measurement of efficiencies.
2.3 The Agency has in place appropriate and robust information management practices to ensure timely and controlled transmission of information, version control and clear ownership of information and data integrity.	2.3.1 Information management roles and responsibilities are clear.
	2.3.2 Information is transmitted and managed in a robust fashion.

Appendix B: Acronyms

Appendix B: Table 2 Acronyms

Acronym	Description
ACS	Assistant Chief Statistician
BP	Business Proposal
BCC	Business Case Costing
CBA	Corporate Business Architecture
CPLD	Corporate Planning Division
CQM	Continuity and Quality Maintenance
CS	Chief Statistician
DAC	Departmental Audit Committee
DG	Director General
DPMF	Departmental Project Management Framework
DPMO	Departmental Project Management Office
FMA	Financial Management Advisor
FPB	Field Planning Board
FPM	Field Portfolio Manager
FRS	Financial Reporting System
Guidelines	The Departmental Project Management Framework Guidelines
IIA	Institute of Internal Auditors
ISPP	Integrated Strategic Planning Process
IT	Information Technology
LTP	Long Term Planning
NSIMS	Non Salary Information Management System
PA	Planning Analyst
PCRA	Project Complexity and Risk Assessment
PM	Project Manager
SMRB	Senior Management Review Board
TMS	Time Management System
TBS	Treasury Board Secretariat

To obtain a copy of any of the following documentation, contact Client Services (613-951-1746; fax: 613-951-0792; hd-ds@statcan.gc.ca).

• CCHS 2013 Rapid Response on Access and Waiting Times – Questionnaire
• CCHS 2013 Rapid Response on Access and Waiting Times – Complement to the User Guide

North American Industry Classification System (NAICS) Canada

NAICS Canada 2022

• Notice of release of the North American Industry Classification System (NAICS) Canada 2022 Version 1.0 January 27, 2022
• What We Heard Report, NAICS Canada 2022 Consultation April 20, 2021

NAICS Canada 2017

• Participate in the revision of the North American Industry Classification System (NAICS) Canada Updated on: December 20, 2019
• Notice of release of North American Industry Classification System (NAICS) Canada 2017 Version 2.0 March 6, 2017
• Notice of changes to the North American Industry Classification System (NAICS) 2017 Version 1.0 December 16, 2016

NAICS Canada 2012

• Revision of the North American Industry Classification System (NAICS) Canada 2012 July 30, 2013
• Notice of discontinuation of the Industry Classification Coding System (ICCS) January 11, 2012
• Notice of provisional NAICS changes for 2012 March 1, 2011

NAICS Canada 2007

• Revision of the North American Industry Classification System for 2012Archived March 9, 2010
• Revision of the North American Industry Classification SystemArchived July 3, 2008