How ranking and recommendations work

The weights can be set by hand or learned from data (see the machine learning part). Scores are usually only meaningful for comparison: a score of 0.8 does not mean 80 percent of anything. Real systems often work in two stages: a cheap method picks a few hundred candidates from millions, then a more expensive model ranks only those. Only the top of the list is ever seen, so small differences near the top matter most.

Each person or item becomes a vector, a list of numbers. Cosine similarity measures the angle between two vectors: it is 1 when they point the same way, 0 when they are unrelated (at right angles), and negative when they point in opposite directions. It ignores length, so someone who rates everything high and someone who rates everything low can still match in direction. Euclidean distance, the straight-line gap, is another common choice. With many items, vectors are mostly empty (most people rate few films), so systems store only the non-empty entries.

This counting idea is part of collaborative filtering: use the behaviour of many users to predict one user's taste, without looking at what the items are. Content-based filtering does the opposite: recommend items whose own features resemble things you liked. A popular collaborative approach, matrix factorisation, imagines a big table of users against items with many blanks, and finds a small set of hidden "taste dimensions" for every user and item such that combining them approximately fills in the table. The blanks are then predicted. Hybrids combine both families.

In the standard PageRank formulation, at each step the surfer follows a random link with probability d (the damping factor, commonly chosen near 0.85 in descriptions of the method) and otherwise jumps to a random page. The jump handles pages with no outgoing links and prevents the surfer being trapped in a small cluster. The scores are the steady-state probabilities of this process. They can be found by power iteration: start with equal scores and repeatedly apply the update, which converges, so the changes shrink until they are negligible. The same idea applies to any network of links, not just web pages.

Popularity bias is the tendency to over-recommend what is already popular. Training data reflects past behaviour, so it can carry human and historical bias forward, and groups that are under-represented in the data may be served worse. Mitigations include adding exploration (occasionally showing less certain items), diversifying results, and auditing outcomes by group.

Explainability helps users and reviewers: "recommended because you watched X" is easier to check than an unexplained score. Recommenders also use personal data, so data minimisation, clear consent and letting users reset or remove their history are good practice. Privacy law varies by place and I do not give legal advice here.

Offline evaluation uses historical data and standard measures such as precision at k (how many of the top k were relevant) and ranking quality measures that reward putting relevant items first. Its weakness is that the history was itself shaped by an earlier system. Online evaluation measures real behaviour but needs care: short-term clicks can reward attention-grabbing items over useful ones, so teams also track longer-term signals such as satisfaction, returns and complaints. No single number captures "good".