Ranking has recently attracted much attention, conceptually
as well as algorithmically and in its uses in major Internet search engines.
A core goal of ranking and related techniques on the Web
is to help people find what they are looking for --
and (depending on the application) to suggest to them things
that they didn't know they were looking for,
but might still find interesting.
Any evaluation of these techniques should therefore consider
such deployment scenarios. The paper starts from
the observation that search engines and recommender systems
generally provide users with some ranking on resources,
and that standard evaluations rest on a comparison of this system output
with an assumed mental representation of the user's ``true ranking''.
This is followed by an overview of i) where and how ranking is used by the
operators of a Web site or similar service, ii) how ranking is used
by the end users of that site or service,
and how such usage is measured, and iii) how and according to which
criteria this usage and the success as well as the quality of ranking are measured.
The paper demonstrates how an interdisciplinary approach can sharpen the
view of challenges and promises of this user-oriented analysis of
intelligent information access.