Programs in Physics & Physical Chemistry
|[Licence| Download | New Version Template] aery_v1_0.tar.gz(71 Kbytes)|
|Manuscript Title: Visualization of the significance of Receiver Operating Characteristics based on confidence ellipses|
|Authors: Nicholas V. Sarlis, Stavros-Richard G. Christopoulos|
|Program title: VISROC.f|
|Catalogue identifier: AERY_v1_0|
Distribution format: tar.gz
|Journal reference: Comput. Phys. Commun. 185(2014)1172|
|Programming language: FORTRAN.|
|Computer: Any computer supporting a GNU FORTRAN compiler.|
|Operating system: Linux, MacOS, Windows.|
|Keywords: Receiver Operating Characteristics (ROC), Significance level, p-value.|
|PACS: 05.40.-a, 05.45.Tp, 89.75.Da, 91.30.Dk.|
|Classification: 4.13, 9, 14.|
Nature of problem:
The Receiver Operating Characteristics (ROC) is used for the evaluation of prediction methods in various disciplines like meteorology, geophysics, complex system physics, medicine etc. The estimation of the significance of a binary prediction method, however, remains a cumbersome task and is usually done by repeating the calculations by Monte Carlo. The FORTRAN code provided here simplifies this problem by evaluating the significance of binary predictions for a family of ellipses which are based on confidence ellipses and cover the whole ROC space.
Using the statistics of random binary predictions for a given value of the predictor threshold εt, one can construct the corresponding confidence ellipses. The envelope of these corresponding confidence ellipses is estimated when εt varies from 0 to 1. This way a new family of ellipses is obtained, named k-ellipses, which covers the whole ROC plane and leads to a well defined Area Under the Curve (AUC). For the latter quantity, Mason and Graham  has shown that it follows the Mann-Whitney U-statistics which can be applied for the estimation of the statistical significance of each k-ellipse. As the transformation is invertible, any point on the ROC plane corresponds to a unique value of k, thus to a unique p-value to obtain this point by chance. The present FORTRAN code provides this p-value field on the ROC plane as well as the k-ellipses corresponding to the (p=)10%, 5% and 1% significance levels using as input the number of the positive (P) and negative (Q) cases to be predicted.
In some machines, the compiler directive -O2 or -O3 should be used to avoid NaN's in some points of the p-field along the diagonal.
Depending on the application, e.g., 4s for an Intel(R) Core(TM)2 CPU E7600 at 3.06GHz with 2GB RAM for the examples presented here
|||S.J. Mason, N.E. Graham, Quart. J.R. Meteor. Soc. 128 (2002) 2145.|
|||H.B. Mann, D.R. Whitney, Ann. Math. Statist. 18 (1947) 50. Program Summary section.|
|||L.C. Dinneen, B.C. Blakesley, J. R. Stat. Soc. Ser. C Appl. Stat. 22 (1973) 269. the end of|
|Disclaimer | ScienceDirect | CPC Journal | CPC | QUB|