Computer Physics Communications Program LibraryPrograms in Physics & Physical Chemistry |

[Licence| Download | New Version Template] aery_v1_0.tar.gz(71 Kbytes) | ||
---|---|---|

Manuscript Title: Visualization of the significance of Receiver Operating Characteristics based on confidence ellipses | ||

Authors: Nicholas V. Sarlis, Stavros-Richard G. Christopoulos | ||

Program title: VISROC.f | ||

Catalogue identifier: AERY_v1_0Distribution format: tar.gz | ||

Journal reference: Comput. Phys. Commun. 185(2014)1172 | ||

Programming language: FORTRAN. | ||

Computer: Any computer supporting a GNU FORTRAN compiler. | ||

Operating system: Linux, MacOS, Windows. | ||

RAM: 1Mbyte | ||

Keywords: Receiver Operating Characteristics (ROC), Significance level, p-value. | ||

PACS: 05.40.-a, 05.45.Tp, 89.75.Da, 91.30.Dk. | ||

Classification: 4.13, 9, 14. | ||

Nature of problem:The Receiver Operating Characteristics (ROC) is used for the evaluation of prediction methods in various disciplines like meteorology, geophysics, complex system physics, medicine etc. The estimation of the significance of a binary prediction method, however, remains a cumbersome task and is usually done by repeating the calculations by Monte Carlo. The FORTRAN code provided here simplifies this problem by evaluating the significance of binary predictions for a family of ellipses which are based on confidence ellipses and cover the whole ROC space. | ||

Solution method:Using the statistics of random binary predictions for a given value of the predictor threshold ε, one can construct the corresponding confidence ellipses. The envelope of these corresponding confidence ellipses is estimated when _{t}ε varies from 0 to 1. This way a new family of ellipses is obtained, named _{t}k-ellipses, which covers the whole ROC plane and leads to a well defined Area Under
the Curve (AUC). For the latter quantity, Mason and Graham [1] has shown that
it follows the Mann-Whitney U-statistics[2] which can be applied[3] for the estimation of the statistical significance of each k-ellipse. As the transformation is invertible, any point on the ROC plane corresponds to a unique value of k, thus to a unique p-value to obtain this point by chance. The present FORTRAN code provides this p-value field on the ROC plane as well as the k-ellipses corresponding to the (p=)10%, 5% and 1% significance levels using as input the number of the positive (P) and negative (Q) cases to be predicted. | ||

Unusual features:In some machines, the compiler directive -O2 or -O3 should be used to avoid NaN's in some points of the p-field along the diagonal. | ||

Running time:Depending on the application, e.g., 4s for an Intel(R) Core(TM)2 CPU E7600 at 3.06GHz with 2GB RAM for the examples presented here | ||

References: | ||

[1] | S.J. Mason, N.E. Graham, Quart. J.R. Meteor. Soc. 128 (2002) 2145. | |

[2] | H.B. Mann, D.R. Whitney, Ann. Math. Statist. 18 (1947) 50. Program Summary section. | |

[3] | L.C. Dinneen, B.C. Blakesley, J. R. Stat. Soc. Ser. C Appl. Stat. 22 (1973) 269. the end of |

Disclaimer | ScienceDirect | CPC Journal | CPC | QUB |