I am an Applied Researcher at Microsoft Research Cambridge. I'm in the Machine Learning and Perception group, in the area of Information Retrieval. Although I'm located in MSR Cambridge, I work for the Microsoft Search Engine Bing.
New: I'm a coordinator of the TREC-2009 Web Track, which evaluates search relevance on a 1 billion page web crawl [ ClueWeb09 ]. This year we are conducting a "TREC adhoc" evaluation and a diversity-aware evaluation.
Workshop: Web Search Click Data 2009 was held in conjunction with WSDM2009 in Barcelona, February 2009.
Research Overview
I am interested in Web search evaluation. I built the VLC, VLC2, WT2g and .GOV test collections, which have been made available to research groups around the world. David Hawking and I coordinated the TREC Web Track experiments. Some publications: Book chapter preprint (pdf), IR'01 (citeseer) and CSIRO'01 (pdf).
I also work on effective Web search, which means making use of information in pages, link structure and URL structure to generate more useful Web search results. Some papers: SIGIR'05 (pdf), SIGIR'01 (pdf), TOIS'03 (pdf) (copying is by permission of ACM, Inc.) and ADCS'03 (pdf).
My PhD was in distributed information retrieval (thesis pdf) which means building a system on top of multiple engines/databases that already exist. My recent work in the area has considered whether (or when) DIR is really practical. Some papers: ADC'99 (ps), DL'00 (pdf), ADC'03 (pdf) and ADC'04 (pdf).
2009
Dennis Fetterly, Nick Craswell, and Vishwa Vinay, The Impact of Crawl Policy on Web Search Effectiveness, in Proceedings of the 32nd annual international ACM SIGIR conference on Research and development in information retrieval, Association for Computing Machinery, Inc., 19 July 2009
Dennis Fetterly, Nick Craswell, and Vishwa Vinay, Measuring the Search Effectiveness of a Breadth-First Crawl , in Proceedings of the 31st European Conference on Information Retrieval (ECIR) , Springer Verlag, 9 April 2009
Marijn Koolen, Gabriella Kazai, and Nick Craswell, Wikipedia Pages as Entry Points for Book Search, in Proceedings of the Second ACM International Conference on Web Search and Data Mining (WSDM'09), Association for Computing Machinery, Inc., February 2009
Emine Yilmaz, Milad Shokouhi, Nick Craswell, and Stephen Robertson, Incorporating user behavior information in IR evaluation, in Understanding the user - Logging and interpreting user interactions in information retrieval. Workshop in Conjunction with the ACM SIGIR Conference on Information Retrieval., 2009
2008
Marc Najork and Nick Craswell, Efficient and Effective Link Analysis with Precomputed SALSA Maps, in 17th ACM Conference on Information and Knowledge Management (CIKM), Association for Computing Machinery, Inc., October 2008
Yunbo Cao, Jingjing Liu, Shenghua Bao, Hang Li, and Nick Craswell, A Two-Stage Model for Expert Search, no. MSR-TR-2008-143, October 2008
Onno Zoeter, Michael Taylor, Ed Snelson, John Guiver, Nick Craswell, and Martin Szummer, A Decision Theoretic Framework for Ranking using Implicit Feedback, in SIGIR 2008 Workshop on Learning to Rank for Information Retrieval, July 2008
Martin Szummer and Nick Craswell, Behavioral Classification on the Click Graph, in World Wide Web Conference, April 2008
Peter Bailey, Nick Craswell, Ian Soboroff, Paul Thomas, Arjen P. de Vries, and Emine Yilmaz, Relevance assessment: are judges exchangeable and does it matter, in SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, ACM, New York, NY, USA, 2008
Dennis Fetterly, Nick Craswell, and Vishwa Vinay, Search Effectiveness with a Breadth-First Crawl, in Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, 2008
Arjen P. Vries, Anne-Marie Vercoustre, James A. Thom, Nick Craswell, and Mounia Lalmas, Overview of the INEX 2007 Entity Ranking Track, pp. 245–251, Springer-Verlag, Berlin, Heidelberg, 2008
Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey, An experimental comparison of click position-bias models, in WSDM '08: Proceedings of the international conference on Web search and web data mining, ACM, New York, NY, USA, 2008
2007
Nick Craswell and Martin Szummer, Random walks on the click graph, in Proceedings of SIGIR 2007, July 2007
Jun Xu, Yunbo Cao, Hang Li, Nick Craswell, and Yalou Huang, Searching Documents Based on Relevance and Type, in ECIR 2007, 2007
Peter Bailey, Nick Craswell, Ian Soboroff, and Arjen P. de Vries, The CSIRO enterprise search test collection, in SIGIR Forum, vol. 41, no. 2, pp. 42–45, ACM, New York, NY, USA, 2007
2006
Michael Taylor, Hugo Zaragoza, Nick Craswell, Stephen Robertson, and Chris Burges, Optimisation methods for ranking functions with multiple parameters, in Proceedings of CIKM 2006, ACM Press, January 2006
Thanh Tang, Nick Craswell, David Hawking, Kathleen Griffiths, and Helen Christensen, Quality and Relevance of Domain-specific Search: A Case Study in Mental Health, in Information Retrieval, vol. 9, no. 2, pp. 207–225, 2006
2005
Charles Clarke, Nick Craswell, and Ian Soboroff, The TREC terabyte retrieval track, in SIGIR Forum, vol. 39, no. 1, pp. 25–25, 2005
Nick Craswell, Stephen Robertson, Hugo Zaragoza, and Michael Taylor, Relevance weighting for query independent evidence, in Proceedings of ACM SIGIR 2005, ACM Press, Salvador, Brazil, January 2005
Nick Craswell, Hugo Zaragoza, and Stephen Robertson, Microsoft Cambridge at TREC-14: Enterprise Track, in Proceedings of TREC-2005, 2005
David Hawking and Nick Craswell, Very Large Scale Retrieval and Web Search, in Chapter in TREC: Experiment and Evaluation in Information Retrieval, MIT Press, 2005
Thanh Tang, David Hawking, Nick Craswell, and Kathleen Griffiths, Focused crawling for both topical relevance and quality of medical information, in Proceedings of CIKM 2005, Bremen, Germany, 2005
2004
Charles Clarke, Nick Craswell, and Ian Soboroff, Overview of the TREC-2004 Terabyte Track, in Proceedings of TREC-2004, Gaithersburg, Maryland USA, November 2004
Nick Craswell and David Hawking, Overview of the TREC-2004 Web Track, in Proceedings of TREC-2004, Gaithersburg, Maryland USA, November 2004
Hugo Zaragoza, Nick Craswell, Michael Taylor, Suchi Saria, and Stephen Robertson, Microsoft Cambridge at TREC 2004: Web and HARD track, in Proceedings of TREC-2004, Gaithersburg, MD: NIST, January 2004
Yasuhiro Yamada, Nick Craswell, Tetsuya Nakatoh, and Sachio Hirokawa, Testbed for Information Extraction from Deep Web, in WWW2004 Poster Proceedings, New York, USA, 2004
Nick Craswell, Francis Crimmins, David Hawking, and Alistair Moffat, Performance and Cost Tradeoffs in Web Search, in Proceedings of the Australasian Database Conference ADC2004, Dunedin, New Zealand, January 2004
David Hawking, Nick Craswell, Francis Crimmins, and Trystan Upstill, How Valuable is External Link Evidence when Searching Enterprise Webs?, in Proceedings of the Australasian Database Conference ADC2004, Dunedin, New Zealand, January 2004
David Hawking, Trystan Upstill, and Nick Craswell, Toward Better Weighting of Anchors, in Proceedings of ACM SIGIR 2004 (poster abstract), Sheffield UK, 2004
2003
Trystan Upstill, Nick Craswell, and David Hawking, Predicting Fame and Fortune: PageRank or Indegree?, in Proceedings of the Australasian Document Computing Symposium, ADCS2003, Canberra, Australia, December 2003
Nick Craswell, David Hawking, Ross Wilkinson, and Mingfang Wu, Overview of the TREC-2003 Web Track, in Proceedings of TREC-2003, Gaithersburg, Maryland USA, November 2003
Nick Craswell, David Hawking, Trystan Upstill, Alistair McLean, Ross Wilkinson, and Mingfang Wu, TREC12 Web and Interactive Tracks at CSIRO, in Proceedings of TREC-2003, Gaithersburg, Maryland USA, November 2003
Peter Bailey, Nick Craswell, and David Hawking, Engineering a multi-purpose test collection for Web retrieval experiments, in Information Processing and Management, vol. 39, no. 6, pp. 853-871, 2003
Trystan Upstill, Nick Craswell, and David Hawking, Query-independent evidence in home page finding, in ACM Transactions on Information Systems (TOIS), vol. 21, no. 3, pp. 286–313, ACM Press, 2003
Ian Soboroff, Ellen Voorhees, and Nick Craswell, Summary of the SIGIR 2003 workshop on defining evaluation methodologies for terabyte-scale test collections, in SIGIR Forum, vol. 37, no. 2, pp. 55–58, ACM Press, 2003
Jared Cope, Nick Craswell, and David Hawking, Automated Discovery of Search Interfaces on the Web, in The Fourteenth Australasian Database Conference, Adelaide, Australia, 2003
2002
Nick Craswell, David Hawking, James Thom, Trystan Upstill, Ross Wilkinson, and Mingfang Wu, TREC11 Web and Interactive Tracks at CSIRO, in Proceedings of TREC-2002, Gaithersburg, Maryland USA, November 2002
Nick Craswell and David Hawking, Overview of the TREC-2002 Web Track, in Proceedings of TREC-2002, Gaithersburg, Maryland USA, November 2002
David Hawking, Nick Craswell, Francis Crimmins, and Trystan Upstill, Enterprise search: What works and what doesn't, in Proceedings of the Infonortics Search Engines Meeting, San Francisco, April 2002
Nick Craswell, David Hawking, Alexander Krumpholz, Ian Mathieson, James A. Thom, Anne-Marie Vercoustre, Peter Wilkins, and Mingfang Wu, XML Document Retrieval with PADRE, in 7th Australasian Document Computing Symposium, Sydney, Australia, 2002
Anne-Marie Vercoustre, James A. Thom, Alexander Krumpholz, Ian Mathieson, Peter Wilkins, Mingfang Wu, Nick Craswell, and David Hawking, CSIRO INEX experiments: XML search using PADRE, in INEX 2002 Workshop, Dagstuhl, Germany, 2002
Trystan Upstill, Nick Craswell, and David Hawking, Buying bestsellers online: A case study in Search & Searchability, in 7th Australasian Document Computing Symposium, Sydney, Australia, 2002
2001
Nick Craswell, David Hawking, Ross Wilkinson, and Mingfang Wu, TREC10 Web and Interactive Tracks at CSIRO, in Proceedings of TREC-2001, Gaithersburg, Maryland USA, November 2001
David Hawking and Nick Craswell, Overview of the TREC-2001 Web Track, in Proceedings of TREC-2001, Gaithersburg, Maryland USA, November 2001
David Hawking, Nick Craswell, and Kathleen Griffiths, Which search engine is best at finding online services?, in Poster Proceedings of WWW10, Hong Kong, May 2001
Trystan Upstill, Raj Nagappan, and Nick Craswell, Visual Clustering of Image Search Results, in SPIE Visual Data Exploration and Analysis VIII, San Jose, California USA, 2001
Nick Craswell, David Hawking, and Stephen Robertson, Effective site finding using link anchor information, in Proceedings of ACM SIGIR 2001, New Orleans, 2001
Nick Craswell, David Hawking, and Kathleen Griffiths, Which search engine is best at finding airline site home pages?, no. 01/45, 2001
David Hawking, Nick Craswell, Peter Bailey, and Kathleen Griffiths, Measuring search engine quality, in Journal of Information Retrieval, vol. 4, no. 1, pp. 33-59, 2001
Nick Craswell, David Hawking, Anne-Marie Vercoustre, and Peter Wilkins, Panoptic Expert: Searching for experts not just for documents, in Ausweb Poster Proceedings, Queensland, Australia, 2001
2000
Nick Craswell, Peter Bailey, and David Hawking, Server Selection on the World Wide Web, in Proceedings of the ACM Digital Libraries Conference, San Antonio, Texas USA, June 2000
Peter Bailey, Nick Craswell, and David Hawking, Chart of darkness: Mapping a large intranet, 2000
David Hawking, Peter Bailey, and Nick Craswell, An intranet reality check for TREC ad hoc, 2000
David Hawking, Peter Bailey, and Nick Craswell, Efficient and flexible search using text and metadata, 2000
Peter Bailey, Nick Craswell, and David Hawking, Dark matter on the Web, in WWW-9 Poster Proceedings, Amsterdam, The Netherlands, 2000
Nick Craswell, Methods for Distributed Information Retrieval, 2000
1999
David Hawking, Peter Bailey, and Nick Craswell, ACSys TREC-8 experiments, in Proceedings of TREC-8, Gaithersburg, Maryland USA, November 1999
David Hawking, Ellen Voorhees, Nick Craswell, and Peter Bailey, Overview of TREC-8 Web track, in Proceedings of TREC-8, Gaithersburg, Maryland USA, November 1999
David Hawking, Nick Craswell, Paul Thistlewaite, and Donna Harman, Results and challenges in Web search evaluation, Elsevier, Toronto, Canada, 1999
Nick Craswell, David Hawking, and Paul Thistlewaite, Merging Results from Isolated Search Engines, in Proceedings of the 10th Australasian Database Conference, Springer-Verlag, Auckland, NZ, 1999
Nick Craswell, Peter Bailey, and David Hawking, Is it fair to evaluate Web systems using TREC ad hoc methods?, in ACM SIGIR '99 Workshop on Web Retrieval, Berkeley, California USA, 1999
1998
David Hawking, Nick Craswell, and Paul Thistlewaite, ACSys TREC-7 experiments, in TREC-7, NIST special publication 500-242, Gaithersburg, Maryland USA, 1998
David Hawking, Nick Craswell, and Paul Thistlewaite, Overview of TREC-7 Very Large Collection Track, in TREC-7, NIST special publication 500-242, Gaithersburg, Maryland USA, 1998
1997
Nick Craswell, Jason Haines, Brendan Humphreys, Chris Johnson, and Paul Thistlewaite, Aglets: A good idea for spidering?, in 4th IDEA Workshop Proceedings, Queensland, Australia, 1997
David Hawking, Paul Thistlewaite, and Nick Craswell, ANU/ACSys TREC-6 experiments, in TREC-6, NIST special publication 500-242, Gaithersburg, Maryland USA, 1997
