Elsevier Science Home
Computer Physics Communications Program Library
Full text online from Science Direct
Programs in Physics & Physical Chemistry
CPC Home

[Licence| Download | New Version Template] aery_v1_0.tar.gz(71 Kbytes)
Manuscript Title: Visualization of the significance of Receiver Operating Characteristics based on confidence ellipses
Authors: Nicholas V. Sarlis, Stavros-Richard G. Christopoulos
Program title: VISROC.f
Catalogue identifier: AERY_v1_0
Distribution format: tar.gz
Journal reference: Comput. Phys. Commun. 185(2014)1172
Programming language: FORTRAN.
Computer: Any computer supporting a GNU FORTRAN compiler.
Operating system: Linux, MacOS, Windows.
RAM: 1Mbyte
Keywords: Receiver Operating Characteristics (ROC), Significance level, p-value.
PACS: 05.40.-a, 05.45.Tp, 89.75.Da, 91.30.Dk.
Classification: 4.13, 9, 14.

Nature of problem:
The Receiver Operating Characteristics (ROC) is used for the evaluation of prediction methods in various disciplines like meteorology, geophysics, complex system physics, medicine etc. The estimation of the significance of a binary prediction method, however, remains a cumbersome task and is usually done by repeating the calculations by Monte Carlo. The FORTRAN code provided here simplifies this problem by evaluating the significance of binary predictions for a family of ellipses which are based on confidence ellipses and cover the whole ROC space.

Solution method:
Using the statistics of random binary predictions for a given value of the predictor threshold εt, one can construct the corresponding confidence ellipses. The envelope of these corresponding confidence ellipses is estimated when εt varies from 0 to 1. This way a new family of ellipses is obtained, named k-ellipses, which covers the whole ROC plane and leads to a well defined Area Under the Curve (AUC). For the latter quantity, Mason and Graham [1] has shown that it follows the Mann-Whitney U-statistics[2] which can be applied[3] for the estimation of the statistical significance of each k-ellipse. As the transformation is invertible, any point on the ROC plane corresponds to a unique value of k, thus to a unique p-value to obtain this point by chance. The present FORTRAN code provides this p-value field on the ROC plane as well as the k-ellipses corresponding to the (p=)10%, 5% and 1% significance levels using as input the number of the positive (P) and negative (Q) cases to be predicted.

Unusual features:
In some machines, the compiler directive -O2 or -O3 should be used to avoid NaN's in some points of the p-field along the diagonal.

Running time:
Depending on the application, e.g., 4s for an Intel(R) Core(TM)2 CPU E7600 at 3.06GHz with 2GB RAM for the examples presented here

[1] S.J. Mason, N.E. Graham, Quart. J.R. Meteor. Soc. 128 (2002) 2145.
[2] H.B. Mann, D.R. Whitney, Ann. Math. Statist. 18 (1947) 50. Program Summary section.
[3] L.C. Dinneen, B.C. Blakesley, J. R. Stat. Soc. Ser. C Appl. Stat. 22 (1973) 269. the end of