Professor David J. Hand

Professor David J. Hand

Professor David J. Hand OBE FBA

Senior Research Investigator and Emeritus Professor of Mathematics

Imperial College, London

______

Professor David Hand is Senior Research Investigator and Emeritus Professor of Mathematics at Imperial College, London, where he formerly held the Chair in Statistics. He is a Fellow of the British Academy, and an Honorary Fellow of the Institute of Actuaries, and has served (twice) as President of the Royal Statistical Society. He is a non-executive director of the UK Statistics Authority, a member of the European Statistical Advisory Committee, a member of the International Scientific Advisory Committee of the Canadian Statistical Sciences Institute, and of the Advisory Board of the Cambridge Institute for the Mathematics of Information. He has published 300 scientific papers and 29 books, including Principles of Data Mining, Information Generation, Measurement Theory and Practice, The Improbability Principle, and The Wellbeing of Nations. In 2002 he was awarded the Guy Medal of the Royal Statistical Society, and in 2012 he and his research group won the Credit Collections and Risk Award for Contributions to the Credit Industry. He was awarded the George Box Medal in 2016. In 2013 he was made OBE for services to research and innovation.

______

For some interviews and articles about my work see:

The Improbability Principle

- Washington Post

- Scientific American

- NPR

- AEON

- Online Bingo

- New York Times

- Slate

- Wisconsin Public Radio

Measurement: A Very Short Introduction

- The British Academy

The Wellbeing of Nations

- Statslife

- Royal Statistical Society, with Paul Allin

Big data

- Big data

- National statistics and data

- Schrödinger Lecture

H Measure

To download information relating to the H-measure, including instructions on accessing R code, go to

http://www.hmeasure.net

______

Honours:

·  Elected Member of the International Statistical Institute, 1988

·  Honorary Fellow of the Institute of Actuaries 1999

·  Thomas L. Saaty Prize for Applied Advances in the Mathematical and Management Sciences, 2001

·  Royal Statistical Society, Guy Medal in Silver, 2002

·  Fellow of the British Academy, elected 2003

·  IEEE ICDM award for Outstanding Contributions, 2004

·  R.C.Bose Memorial Keynote Address, 2005

·  President of the International Federation of Classification Societies, 2006 and 2007

·  Forum for Interdisciplinary Mathematics award for Outstanding Contributions to the Mathematical Sciences, 2005

·  Royal Society Wolfson Research Merit Award, 2006-2009

·  President, Royal Statistical Society, 2008 and 2009, 2010

·  Credit Collections and Risk award for Contributions to the Credit Industry, 2012

·  OBE, New Year's Honours, 2013

·  George Box Medal, 2016

Books:

·  Hand D.J. (2016) Measurement: A Very Short Introduction. Oxford University Press.

·  Allin P. and Hand D.J. (2014) The Wellbeing of Nations: Meaning, Motive, and Measurement. Wiley.

·  Hand D.J. (2014) The Improbability Principle: Why Coincidences, Miracles, and Rare Events Happen Every Day. Farrar, Straus, and Giroux.

·  Krzanowski W.J. and Hand D.J. (2009) ROC curves for continuous data. CRC/Chapman and Hall.

·  Hand D.J. (2008) Statistics: a very short introduction. Oxford University Press.

·  Hand D.J. (2007) Information Generation: How data rule our world. OneWorld Publications.

·  Hand D.J. and Herzberg A.M. (eds) (2005) Selected statistical papers of Sir David Cox. CUP.

·  Hand D.J. (2004) Measurement theory and practice: the world through quantification, Edward Arnold.

·  Adams N.M., Crowder, M..J., Hand D.J., and Stephens D.A. (eds.) (2004) Methods and models in statistics: in honour of Professor John Nelder, FRS. World Scientific.

·  Hand D.J., Adams N.M., and Bolton R.J. (eds.) (2002) Pattern detection and discovery. Springer.

·  Hand D.J., Mannila H., and Smyth P. (2001) Principles of data mining, MIT Press.

·  Hoffmann F., Hand D.J., Adams N., Fisher D., and Guimaraes G. (eds) (2001) Advances in Intelligent Data Analysis. Springer.

·  Hand D.J., Kok J.N., and Berthold M.R. (eds.) (1999) Advances in intelligent data analysis. Springer-Verlag.

·  Berthold M. and Hand D.J. (eds) (1999) Intelligent data analysis. Springer-Verlag.

·  Hand D.J. and Jacka S. (eds.) (1998) Statistics in finance, Edward Arnold.

·  Hand D.J. (1997) Construction and assessment of classification rules, Wiley.

·  Hand D.J. and Crowder M.J. (1996) Practical longitudinal data analysis, Chapman & Hall.

·  Gower J.C. and Hand D.J. (1996) Biplots, Chapman & Hall.

·  Daly F., Hand D.J., Jones M.C., Lunn A.D., and McConway K. (1995) Elements of statistics, Addison-Wesley.

·  Hand D.J., Daly F., Lunn A.D., McConway K. and Ostrowski E. (eds.) (1994) A handbook of small data sets, Chapman & Hall.

·  Hand D.J. (ed.) (1994) AI and computer power: the impact on statistics, Chapman & Hall.

·  Hand D.J. (ed ) (1993) Artificial intelligence frontiers in statistics, Chapman & Hall,

·  Crowder M.J. and Hand D.J. (1990) Analysis of repeated measures, Chapman & Hall.

·  Hand D.J. and Everitt B.S. (eds.) (1987) The statistical consultant in action, Cambridge University Press.

·  Hand D.J. and Taylor C.C. (1987 Multivariate analysis of variance and repeated measures: a practical guide for behavioural scientists, Chapman & Hall.

·  Hand D.J. (1985) Artificial intelligence and psychiatry, Cambridge University Press [Fench translation, 1991]

·  Hand D.J. (1982) Kernel discriminant analysis, Research Studies Press/Wiley.

·  Everitt B.S. and Hand D.J. (1981) Finite mixture distributions, Chapman & Hall.

·  Hand D.J. (1981) Discrimination and classification, Wiley.

Papers: (Please email me if you would like pdf files)

Papers in refereed journals

R179: Hand D.J. (2018) Statistical challenges of administrative and transaction data (with discussion). Journal of the Royal Statistical Society, Series A, 181, 1-24.

R178: Hand D.J. and Christen P. (2017) A note on using the F-measure for evaluating record linkage algorithms. Statistics and Computing, 19, DOI 10.1007/s11222-017-9746-6. Spiral http://hdl.handle.net/10044/1/46235

R177: Allin P.W. and Hand D.J. From a system of national accounts to a process of national wellbeing accounting. Accepted by International Statistical Review.

R176: Batchelor B.G. and Hand D.J. (1976) Fast Monte Carlo integration of pdf estimators. Journal of Cybernetics, 5, 111-124.

R175: Allin P. and Hand D.J. (2016) New statistics for old? – Measuring the wellbing of the UK (with discussion). Journal of the Royal Statistical Society, Series A, Statistics and Society, 180, 3-43. http://hdl.handle.net/10044/1/28812

R174: Hand D.J. (2016) Big data and data sharing. Journal of the Royal Statistical Society, Series A: Statistics in Society, 179, 1-3, http://hdl.handle.net/10044/1/33934

R173: Hand D.J. (2015) From evidence to understanding: a commentary on Fisher (1922) ‘On the mathematical foundations of theoretical statistics’ by Philosophical Transactions of the Royal Society, A 373, Issue 22039, Article ID 20140252.

R172: Hand D.J. (2014) Wonderful examples, but let’s not close our eyes. Statistical Science, 29, 98-100.

R171: Hand D.J. and Anagnostopoulos C. (2014) A better beta for the H-measure of classification performance. Pattern Recognition Letters, 40, 41-46.

R170: Hand D.J. and Adams N.M. (2013) Selection bias in credit scorecard evaluation. Journal of the Operational Research Society, 65, 408-415.

R169: Pavlidis N.G., Tasoulis D.K., Adams N.M., and Hand D.J. (2012) Adaptive consumer credit classification. Journal of the Operational Research Society, 63, 1645-1654.

R168: Hand D.J. and Anagnostopoulos C. (2013) When is the area under the receiver operating characteristic curve an appropriate measure of classifier performance? Pattern Recognition Letters, 34, 492-496.

R167: Henrion M., Hand D.J., Gandy A., and Mortlock D.J. (2013) CASOS: A subspace method for anomaly detection in high dimensional astronomical databases. Statistical Analysis and Data Mining. 6, 53-72.

R166: Hand D.J. (2012) Assessing the performance of classification methods. International Statistical Review, 80, 400-414.

R165: Anagnostopoulos C., Tasoulis D., Adams N.M., Pavlidis N.G., and Hand D.J. (2012) Online linear and quadratic discriminant analysis with adaptive forgetting for streaming classification. Statistical Analysis and Data Mining, 5, 139-166.

R164: Ross G.J., .Adams N.M., Tasoulis D.K., and Hand D.J. (2012) Exponentially weighted moving average charts for detecting concept drift. Pattern Recognition Letters, 33, 191-198.

R163: Hand D.J. (2011) Discussion of ‘Bayesian models and methods in public policy and government settings’ by S.E.Fienberg. Statistical Science, 26, 227-230.

R162: Bentham J. and Hand D.J. (2012) Data mining from a patient safety database: the lessons learned. Data Mining and Knowledge Discovery, 24, 195-217.

R161: Hand D.J. and Crowder M.J. (2012) Overcoming selectivity bias in evaluating new fraud detection systems for revolving credit operations. International Journal of Forecasting, 28, 216-223.

R160: Henrion M., Mortlock D.J., Hand D.J., and Gandy A. (2011) A Bayesian approach to star-galaxy classification. Monthly Notices of the Royal Astronomical Society, 412, 2286-2302.

R159: Brentnall A.R., Crowder M.J., and Hand D.J. (2011) Approximate repeated-measure shrinkage. Computational Statistics and Data Analysis, 55, 1150-1159.

R158: Pavlidis N., Tasoulis D.K., Adams N.M., and Hand D.J. (2011) λ-perceptron: an adaptive classifier for data streams. Pattern Recognition, 44, 78-96.

R157: Yu K, Ally A.K., Yang S., and Hand D.J. (2010) Kernel quantile-based estimation of expected shortfall. The Journal of Risk, 12, 15-32.

R156: Heard N.A., Weston D.J., Platanioti K., and Hand D.J. (2010) Bayesian anomaly detection methods for social networks. Annals of Applied Statistics, 4, 645-662.

R155: Hand D.J. (2010) Evaluating diagnostic tests: the area under the ROC curve and the balance of errors. Statistics in Medicine, 29, 1502-1510.

R154: Anagnostopoulos C., Adams N.M., Hand D.J. (2010) Streaming covariance selection with applications to adaptive querying in sensor networks. The Computer Journal, doi: 10.1093/comjnl/bxp123, 53, 1402-1414.

R153: Pavlidis N.G., Adams N.M., Nicholson D., and Hand D.J. (2010) Prospects for bandit solutions in sensor management. The Computer Journal, 53, 1370-1383.

R152: Brentnall A.R., Crowder M.J., and Hand D.J. (2010) Likelihood-ratio changepoint features for consumer-behaviour models. Journal of the Operational Research Society, 61, 462-472.

R151: Anagnostopoulos C., Tasoulis, D.K., Adams N.M, and Hand D.J. (2009) Temporally adaptive estimation of logistic classifiers on data streams. Advances in Data Analysis and Classification, 3, 243-261. DOI: 10.1007/s11634-009-0051-x

R150: Hand D.J. and Zhou F. (2010) Evaluating models for classifying customers in retail banking collections. Journal of the Operational Research Society, 61, 1540-1547.

R149: Krzanowski W.J. and Hand D.J. (2011) Testing the difference between two Kolmogorov-Smirnov values in the context of Receiver Operating Characteristic curves. Journal of Applied Statistics, 38, 437-450.

R148: Brentnall A., Crowder M.J., and Hand D.J. (2010) Predictive-sequential forecasting system development for cash machine stocking. International Journal of Forecasting, 26, 764-776.

R147: Hand D.J. (2009) Mining the past to determine the future: rejoinder. International Journal of Forecasting, 25, 461-462.

R146: Tasoulis D., Adams N.M., and Hand D.J. (2010) Selective fusion of out-of-sequence measurements. Information Fusion, 11, 183-191.

R145: Hand D.J. (2009) Measuring classifier performance: a coherent alternative to the area under the ROC curve. Machine Learning, 77, 103-123 , doi:10.1007/s10994-009-5119-5

R144: Jamain A. and Hand D.J. (2009) Where are the large and difficult data sets? Advances in Data Analysis and Classification, 3, 25-38.

R143: Krzanowski W.J. and Hand D.J. (2009) A simple method for screening variables before clustering microarray data. Computational Statistics and Data Analysis, 53, 2747-2753.

R142: Hand D.J. (2009) Modern statistics: the myth and the magic (RSS Presidential Address). Journal of the Royal Statistical Society, Series A, 172, 287-306.

R141: Hand D.J. (2008) Breast cancer diagnosis from proteomic mass spectrometry data: a comparative evaluation. Statistical Applications in Genetics and Molecular Biology. 7, iss.2, article 15. DOI: 10.2202/1544-6115.1435

R140: Hand D.J. (2008) Mining the past to determine the future: problems and possibilities. International Journal of Forecasting, 25, 441-451.

R139: Pei T, Jasra A., Hand D.J., Zhu A-X., and Zhou C. (2008) DECODE: a new method for discovering clusters of different densities in spatial data. Data Mining and Knowledge Discovery. 18, 337-369.

R138: Brentnall A., Crowder, M.J., and Hand D.J. (2010) Predicting the amount individuals withdraw at cash machines. Statistical Modelling, 10, 197-214.

R137: Hand D.J., Brentnall A., and Crowder M.J. (2008) Credit scoring: a future beyond empirical models. Journal of Financial Transformation, 23, 121-128.

R136: Whitrow C., Hand D.J., Juszczak P., Weston D., and Adams N.M. (2008) Transaction aggregation as a strategy for credit card fraud detection. Data Mining and Knowledge Discovery, 18, 30-55.

R135: Juszczak P., Adams N.M., Hand D.J., Whitrow C., and Weston D.J. (2008) Off-the-peg or bespoke classifiers for fraud detection? Computational Statistics and Data Analysis, 52, 4521-4532.

R134: Weston D.J., Hand D.J., Adams N.M., Juszczak P., and Whitrow C. (2008) Plastic card fraud detection using peer group analysis. Advances in Data Analysis and Classification, 2(1), 45-62.

R133: Jamain A. and Hand D.J. (2008) Mining supervised classification performance studies: a meta-analytic investigation. Journal of Classification, 25, 87-112.

R132: Brentnall A.R., Crowder M.J., and Hand D.J. (2008) A statistical model for the temporal pattern of individual ATM withdrawals. Applied Statistics, 57, 43-59.

R131: Krzanowski W.J. and Hand D.J. (2007) A recursive partitioning tool for maximising prediction accuracy. Advances in Data Analysis and Classification, 1, 241-254.

R130: Twala B.E.T.H., Jones M.C., and Hand D.J. (2008) Good methods for coping with missing data in decision trees. Pattern Recognition Letters, 29, 950-956.

R129: Wu X, Kumar V., Quinlan J.R., Ghosh J., Yang Q., Motoda H., McLachlan G.J., Ng A., Liu B., Yu P.S., Zhou Z-H., Steinbach M., Hand D.J., and Steinberg D. Top 10 algorithms in data mining. (2008) Knowledge and Information Systems, 14, 1-37.

R128: Wu I-D. and Hand D.J. (2007) Handling selection bias when choosing actions in retail credit applications. European Journal of Operational Research, 183, 1560-1568.

R127: Hand D.J., Whitrow C., Adams N.M., Juszczak P., and Weston D. (2008) Performance criteria for plastic card fraud detection tools. Journal of the Operational Research Society, 59, 956-962.

R126: Zhang Z.C. and Hand D.J. (2006) Detecting groups of anomalously similar objects in large data sets. Intelligent Data Analysis, 10, 473-483.

R125: Hand D.J., Krzanowski W.J., and Crowder M.J. (2007) Optimal predictive partitioning. Statistics and Computing, 17, 11-23.

R124: Crowder M.J., Hand D.J., and Krzanowski W.J. (2007) On optimal intervention for customer lifetime value. European Journal of Operational Research, 183, 1550-1559.

R123: King M.D., Crowder M.J., Hand D.J., Harris N.G., Williams S.R., Obrenovitch T.P., and Gadian D.G. (2005) Is anoxic depolarisation associated with an ADC threshold? A Markov chain Monte Carlo analysis. NMR in Biomedicine, 18, 587-594.