CRITIQUE OF JUSTICE DEPARTMENT EVALUATION (ED1996-1e)

"A Statistical Analysis of the Impacts of the 1977 Firearms Control Legislation"

Note: This is a copy of a letter of complaint sent to the Auditor General for Canada by a federal public servant who wishes to remain anonymous for now. In preparing his analysis of this Justice Department document, he has been assisted by a colleague who is a statistician. The author has given us permission to use his analysis as we see fit. If it is essential for anyone to talk to the author, I will give him the message and he can then decide if he wishes to respond directly.

For more information please contact:

Garry Breitkreuz, MP (Yorkton-Melville) House of Commons, Ottawa, Ontario K1A 0A6

Phone: (613) 992-4394 Fax: (613) 992-8676 E-Mail: breitkreuzg@reform.ca

December 6, 1996

Dear Sir,

I am writing to you for three reasons; because I believe you have recommended to the Dept of Justice that they evaluate our firearms laws in order to provide assurance to the public; because I have no confidence in the DoJ addressing my concerns in an honest and forthright manner, and I am also writing to you because I believe it is your job to assure the Canadian public that our money is being spent in a responsible and honest manner. I believe it is your role to try to ensure some measure of honesty and integrity to the running of our government . I would like to know your opinion of the recently released report by the DoJ ED1996 - 1e (A Statistical Analysis of the Impact of the 1977 Firearms Control Legislation).

I have previously discussed my assessment of this paper (included in this letter) with a fellow employee (who wishes to remain anonymous) who is an experienced (senior) statistician. He found no major faults with my comments and added that what he additionally found puzzling was that ARIMA analysis are not normally used to demonstrate cause/effect relationships but rather are normally used based on a much longer time frame to make predictions for the future. I don't believe that he was convinced that the ARIMA analysis was appropriate as used in this paper. I would therefore ask if you feel that my following

assessment of this report seems to you a fair and reasonable critique or if my assessment contains glaring shortcomings.

I believe that it is very important for the DoJ to appear to be honest and factual in their assessments of their actions and proposals. It is because of shoddy rewrites of evaluation documents such as ED 1996-1e that I and many other firearms owners are so mistrustful of our Dept. of Justice. I think that it is a national embarrassment when a large group of responsible citizens feel that the DoJ of Canada is dishonest. I am also enclosing an internet published account (which I strongly believe to be factual) in which the Montreal Customs and Surete Quebec appear to have exceeded their authority for personal rather than valid legal reasons. As with my comments re the DoJ, when our legal agencies cease to be seen as honest and enforcing a common law, it becomes extremely difficult to force otherwise honest citizens to obey laws which they consider unnecessary and deceitful.

Assessment of ED 1996 1e

My criticism of this document begins with its authorship. Near the top of page xi it is stated that evaluation policy specifies that evaluations are to be completed by officials independent of the programs yet later on this page it states that this report is prepared by the DoJ (based on Prairie Research Associates work) which to me seems hardly an arms length evaluation. I believe that is why there appears to be so many places in the report where interpretations are deliberately slanted to favour a DoJ point of view and are by no means

impartial assessments. Some examples of this bias are as follows;

a) page xii (near bottom) "there has been a steady decline since 1977" which is true but in western Canada (page 44) the trend started as early as the late 1960's and elsewhere starts about 1975 as stated in the report. I believe that the wording of this page is trying to encourage the impression that the 1977 law (whose significant features actually started January 1979) caused the change rather than "piggy backs" a previously

started trend. I also believe if the law is the critical variable in change then these changes should occur across the country and one should not have anomalies such as Western Canada anticipating the law by 8 or 9 years.

b) page xiii paragraph 2 "robberies with firearms reached a historic low" totally ignores the dramatic rise in the rate of firearm use (in robberies) in Ontario and Western Canada. The Canada total rate appears driven by the Quebec statistics. As before if the changes in firearm laws are having an effect, these changes should be apparent in all regions of the country.

c) page xiii "the law has reduced the rate of firearm accidents in Western Canada"; apart from being a localized correlation(if any) the statement appears to ignore a 30 year trend in such accidents (page 48) which probably is primarily attributable to the implementation of hunter education classes (in B.C. and I assume elsewhere)

d) page xiii "the law has reduced deaths by 55/year" ignores that this same structured analysis showed that more guns (presumably more people obtained FAC's in order to get more guns) reduced the firearm death rate by roughly twice as much based on the "t values" given (page 84 - 85)

e) page xiv "the law shows a relationship with accidental deaths"; this relationship is more or less invisible to my eyes and as stated earlier ignores a dramatic 30 year trend.

I would like to make a number of general observations related to the text of the report, which I will list by page.

1) page 1 at the top says the rationale for the changes to the law was to reduce firearms deaths and injury but at the bottom of the page says that the government has now another theme that of creating a gentler society (which some Liberal Senators have referred to as social engineering). This latter theme of social engineering assumes that a gentler society is inversely related to firearm density and further seems an ill defined term which the report does not address. I am unaware of any widely accepted scientific studies

showing that the absence of responsibly owned firearms assures a peaceful or compassionate society. I

believe this is more a feature of national beliefs, history, economy and social programs.

2) page 6 makes it apparent that the real start of the legislation is Jan. 1979 with the implimentation of the FAC system. Any changes in penalties for the use of a gun in a crime are almost always plea bargained away and where used I understand never exceed a miminum sentence. The significance of the 1979 start of the law is that a 1978 line on the various graphs for exploratory analysis tends to obscure the beginnings of pre-existing trends.

3) page 18 refers to various studies related to alleged causes of firearms deaths; I believe that Killias has now been widely discredited as a scientific author. Footnote 16 for example I believe comments on the relationship of gun density vs firearm deaths while ignoring climatic differences. I believe it is felt (by impartial observers) that high suicide rates in northern Europe are more closely related to short winter day length and by factors other than firearm ownership. Similarly one of the reviewers referred to a paper by Kellerman which has now been widely discredited as comparing two widely different groups of people

in regard to murder rate vs gun presence.

4) page 27 footnote 38 comments on a profound bias present if an important variable is ignored but fails to mention a profound bias is also present if variables are co-related. I am reminded of a study cited when I was a university student (can't remember the publication) by Tinbergen done in the 1950's or early 1960's which showed a relationship between birds of the genus Parus in England (P. major; the great tit and P.niger?; the blue tit) and the abundance of beech mast (fruit of the beech tree we were told) in Holland or

Belgium. The point in relating this example is that there were no beech trees in England. An excellent

relationship, only the causal variable was missing.

5) page 29 the table summarizing structural models fails to note that cross sectional comparisons (between different states or provinces) assumes that the major differences are only the variable being considered. Comparing a state with large, dense, impoverished cities with a rural one for differences in the effect of laws for example makes little sense. In the same table ; time series comparisons assume that all important

variables are included and that there is little possibility of co-related variables of which one or more is not

noted. My comments related to the Tinbergen paper above would be an example of such an error. Similarly I am currently reading (on the computer) of eastern US hunters correlating the abundance of deer and squirrels; the important variable not noted in such a comparison is the abundance of acorns and other nuts. DoJ logic as expressed in paper under consideration would conclude that squirrels limited deer populations (or vice versa).

6) page 33 appears to relate gross income to socio economic status and does not adjust for inflation. I feel that this probably negates any correlation that might exist.

7) page 35 in considering behavioural variables uses the number of drug arrests and appears to assume that there has been no change in the level of enforcement particularly for "soft drugs" such as marijuana. I don't believe such an assumption is valid.

8) page 38 - 39 section 3.4.4 describing the implementation of the law more or less negates any close correlation of the law in this case. Almost any change occurring within 3 - 5 years before or after the date would be accepted as a valid correlation.

9) page 43 the total Canada graph appears to be driven by the statistics for Quebec. The trend in western Canada could be said to start as early as 1968 / 69. I believe that I said above that valid cause/effect relationships should occur across the country and if they don't one has to question the validity of assuming that they are real relationships. Throughout this paper the author(s) appear to be grasping for data to support the implementation of the law as a significant variable and usually ignoring any contrary data.

10) page 46 I find it ironic that the exploratory analysis seems to show a strong correlation of suicides with the law while (p 89) the regression model shows no relationship for total suicides and insignificant relationship for firearms suicides. I won't comment on the results of the ARIMA analysis since its use on accidental deaths (p 61) particularly convinces me that this is a dubious proceedure at least as used in this paper.

11) page 47 - 48 there is no mention of CORE programs starting in the 1960's (perhaps because they are provincial programs). There seems no discussion of why the ARIMA analysis on p 61 shows a signifcant relationship which appears invisible on the page 48 graphs.

12) page 50 as with other cases the graph for the ratio of robbery - firearm use for total Canada seems driven by Quebec and on page 51 there is no mention that the use of firearms in robberies has increased dramatically in Ontario and western Canada in contrast with Quebec.

13) page 51 in addition to my comments re robbery, the real trend in firearm homicides appears to have been since 1974 (except for the Atlantic where I think the total numbers are relatively small). The wording in the report tries to imply (I feel) that the trend is since 1978 and therefore caused by the law. Similarly near the bottom of the page is the statement that a change became a trend after the law was enacted. It is my understanding that trends have beginnings and may have ends but that they don't usually become trends

because of variables in the middle of their duration.

14) page 53 near the bottom; I strongly believe that an ARIMA analysis shows synchrony of change between variables but does not prove cause/effect relationships as appears to be implied in the paper.

15) page 54 - 55 The analysis appears to have used a rather blurred end to the pre law homicide pattern. Statements made here and I think elsewhere leave me with the impression that for the analysis the pattern up to about 1973 was compared to the pattern after 1977, 78, and 1979. I believe that if such was done, it serves to obscure changes which took place in the 3 - 4 years previous to the legislative changes. I believe that most of my concerns with the structural analysis have for the most part been covered in my criticism of the introduction of this paper. I believe that the model stands a very high risk of faulty conclusions because of covariance and because important variables were not included (because they were not measured and perhaps because they were not considered.) I would agree fully with the report that the firearm

fatalities need to be analysed on an individual basis rather than on the changes in rates of aggregate

numbers.

In summary, I am writing to you because I believe it is your job to assure the Canadian public that our money is being spent in a responsible and honest manner. I believe it is your role to try to ensure some measure of honesty and integrity to the running of our government. As an employee at a government research facility, I find impossible to believe that this paper would be accepted as written nor can I believe that a reputable consulting company would have written it in such a slanted and biased fashion. I look

forward to your comments on my critique and will pass them along to the others to whom this letter has been copied.

Thank you

cc. Nanaimo Fish and Game Club

The hon. J. Ramsay

The hon. G. Breitkreutz