.. the estimate of statistical reliability attributable only to sampling can be expressed by stating that there is 95 percent confidence that the reserves lie within 21.5 Tcf. (or about 10 percent) of 225.5 Tcf.

It should be emphasized that contrary to implications by Mr. Halverson, industry personnel played no role in the functions of the basic elements of the NGRS. As stated on page 11 of the NGRS report the procedures adopted were:

The structure which permitted the interplay of the
different elements was organized into the following
groups (Appendix I): 6/

1. Gas field nomenclature problems were resolved
by the FPC staff, (Appendix II) 6 and a com-
prehensive list of all gas fields in the United
States was compiled by the Oil Information Center
(OIC) at the University of Oklahoma Research
Institute (Appendix III). 6/

2.

Independent reserves teams composed of geologists,
engineers and other professional staff members of
the FPC made the field reserves estimates with
assistance, as available, from the United States
Geological Survey which assumed the responsibility
for the Outer Continental Shelf Fields, United
States Department of the Navy, and from colleges
and universities (Appendix IV). 6/

3. The independent accounting agent was selected by
the FPC. This agent. Arthur Young & Company, was
commissioned to (Appendix V):

6/

(a) Provide security for individual field
reserve estimates,

(b) Classify gas fields by reserve size and
age; perform random selection of fields
for reserve estimations as prescribed by
the statistical validation team,

(c) Consolidate the findings of the reserve
teams, and

(d) Report United States gas reserves estimates
to the National Gas Survey.

67 These references relate to the Appendix number in the NGRS report.

4. A sta:istical validation team / consisted
of experts from the Office of Management and
Budget (OMB), the United States Geological
Survey, the Bureau of Census, the FPC end
academic personnel as commissioned by the FPC.
The team had the responsibility of prescribing
sampling proc dures for valid reserve estimation
(Appendix VI). 8/

I emphasize that neither the Supply-Technical Advisory Task Force-Natural Gas Supply nor any other group with industry representation had any role in the development of the statistical sample. The contrary might be inferred from page 2 of Mr. Halverson's letter. It should also be emphasized that the role of technical advisory committees in the National Gas Survey and the administrative procedures governing their activities have been described in detail by Chairman John N. Nassikas and Mr. Gordon Gooch, former General Counsel of the Federal Power Commission. Their testimony before Congressman Neal Smith's Subcommittee on Special Small Business Problems at Hearings held March 16, 1972, should be reviewed by Mr. Halverson. The points he raises concerning the use, formation and management of advisory committees within the National Gas Survey have all been eloquently answered in the testimony given by Chairman Nassikas and Mr. Gooch.

Mr. Halverson seems to have difficulty retaining in his mind the objectives of the GRS. He states, "This suggests that had the purpose of the NGS (sic) been to examine critically the accuracy of AGA figures, the certainty sample would have been drawn from among those fields reported by the AGA to contain relatively small amounts of natural gas. The objective of the NGRS was to obtain an independent estimate of the Nation's natural gas reserves as of December 31, 1970. Even Mr. Halverson himself does not argue the fact. (See page 7 of Mr. Halverson's letter.) Yet he proposes the converse as a premise. As a result of the false premise, the conelusion to this argument, like those addressed to the validity of the statistical analysis procedures, has no relevance and is purely a matter of academic interest. If Mr. Halverson's sampling procedures had been followed, the total estimate of reserves could not have been stated with an acceptable degree of reliability. Based on the number of fields that could have been sampled, the computed 95% confidence intervals

7/ A list of the members of the Statistical Validation Team was given as Exhibit A in my letter to you of July 27, 1973, and as Exhibit A in my letter to Senator Adlai E. Stevenson III of November 20, 1973.

8/

This reference related to the Appendix number in the NGRS
report.

55-817 - 75 pt. 2 38

have been much too wide to be of value. The use of a certainty stratum containing the largest fields was the most efficient sampling technique on the basis of the actual parameters and the realities of the study.

On page 3 of Mr. Halverson's letter, he states that the "meager random sampling . . . leans over backwards" to avoid detecting "A.G.A. under-reporting". At the same time he refers to page 18 of the Report of the Statistical Validation Team (9VT) (NGRS Appendix VI), where it is stated that the failure to adequately sample from certain population strata "introduces the possibility of major error" in estimating total reserves.

In the first place he has not presented any hard facts that show that the sample was really "meager". Second, he has ignored the SVT's statements in that same paragraph on page 18 which states:

Examination of the results of a sample designed
by the SVT to yield minimum sampling error
revealed... the fact that only a few sample
fields occurred in each of the three smallest size
strata
As a consequence of these observations,
it was felt that there was a possibility of making
a major error in estimating the total reserves due
to these strata. The allocation of 50 fields was
altered to mitigate the above difficulties.

In other words, the SVT initially allocated the 50 fields to the smaller field size strata, and then made adjustments in the allocation because the Team felt there were too few sample fields in the smallest three field size strača (0-1 Bcf, 1-5 Bcf, and 5-10 Bcf). The Team increased the number of sample fields in these strata because they felt that the probability was high, that FPC estimates for these fields could be outside the narrow stratum ranges, due perhaps to A.G.A. overestimates or underestimates. In my opinion, it is evidenced on page 18 of the SVT Report that the SVT did not "lean" either forward or backward, but designed a sound objective and realistic sampling procedure consistent with the goals of the NGRS as set forth in the reserves study order.

A third note on this subject is that the statistical prediction of the total reserves in the 0-1 stratum (1,879 fields) was 3,892 billion cubic feet as compared to the population (primarily A.G.A. estimate) of 723 billion cubic feet in this stratum. This is shown in Tables 2 and 5 of the SVT Report. Thus the sample of four fields described as "meager by Mr. Halverson would in fact yield an estimate of reserves in the 0-1 stratum that is five times greater than the A.G.A. estimate. That fact

has not gone unnoticed and the reason for it has been investigated. The difference between the stratum estimates is due almost entirely to a difference in reserves estimates for one field. The cause of the difference in the reserves estimates reflects a difference in judgment between an FPC field team leader and the individual who prepared the A.G.A. estimates. Although I accept the professional judgment of the FPC field team leader, I question that the use of the term underreporting would be appropriate under the circumstances. Isolated and individual examples of A.G.A. underestimates can be cited. It is also true that cases of overestimates of reserves can be cited. However, it is more important to remember that the number and extent of A.G.A. overestimates greatly exceeded the number and extent of A.G.A. underestimates.

One last comment on bias in statistical sampling seems appropriate. Because of his preoccupation with underreporting, Mr. Halverson would bias the sampling procedure, even though he offers no convincing information to indicate that underestimation of proven reserves is more likely to occur than overestimation. On the other hand, the NGRS was an objective technical study aimed at obtaining an independent estimate of reserves. It was not a quest to show that "roven reserves were either overestimated or underestimated.

Mr. Halverson misrepresents one of my answers to a question posed by Mr. Bangert. The question and the pertinent part of the answer from my letter of July 27, 1973, which is part of the record of the Hearings are:

Clearly my statement above refers to percentage discrepancies between field reserves estimates. Mr. Halverson seems to have missed the whole point of my analysis presented in response to Mr. Bangert's question and fails to differentiate between the terms "percentage discrepancies", and "ratios" or "comparisons". My statement above, based on percentage discrepancies, is not in contradiction of my later statement which is based on ratios defined in a precise way (FPC estimate/A.G.A. estimate):

- 15

Without recourse to formal statistical analysis,
the fact that A.G.A. overestimates as defined
above In terms of ratios7 exceed the A.G.A.
underestimates by a factor of greater than four
would, in itself, lend strong support to any
contention that the A.G.A. proven reserves total
is an overestimation.

That statement rings true and even Mr. Halverson does not dispute it.

My contention is that the word "discrepancy" implies that one of the two estimates is in error, whereas the terms "ratio" and "comparison" do not. In my opinion, the field reserves team estimate was the true field reserves estimate and the one I would accept as the standard. That judgment, however, was not made in the NGRS. It should also be noted that the report of the NGRS contains almost 500 pages. There is not to be found on any of those pages a percentage discrepancy between two field reserves estimates nor any other comparison of field reserve figures. Nevertheless, comparisons of total reserves estimates are both valuable and appropriate. An omission of a comparison of the NGRS and the A.G.A. estimates would have been naive. As Mr. Halverson states, there are tables which show both NGRS and A.G.A. estimates of total reserves of gas in the United States under various categories.

Paragraph 2 on page 5 of Mr. Halverson's letter is based on a serious misinterpretation of the Reserves Study Order which has led him to confuse the function of the reserve team supervisor and the field team leaders. The function of the reserve team supervisor was described specifically in the Reserves Study Order as follows:

R.

Independent reserve estimations are
transmitted on a confidential basis
to the reserve team supervisor. The
reserve team supervisor will compare
the independent field reserve esti-
mates from the A.G.A. He may compare
them with any other source ir.cluding
(but not limited to) the following:

1. OIC Data Bank at the Oklahoma
University Research Institute.

2. Natural Gas Companies' Annual

Report of Gas Supply - FPC Form 15.