The Interest Graph is WHAT people care about, not WHO people care about.
A very important aspect of the Interest Graph is that unlike the Social Graph, which is for the most part static (except for an occasional friending or unfriending), the Interest Graph is elastic, dynamic, and rapidly changing for any individual consumer based on:
What a consumer searches for (on Google, Bing, YouTube, Twitter, etc)
What a consumer follows (on Facebook, Twitter, Google +1, Tumblr, Quora, etc)
What a consumer expresses (as blog posts, tweets, pins, stashes, statuses, reviews, pictures, conversations, etc.)
Note that searching is private but following and expressing are, for the most part, public. The Internet-Scale Interest Graph therefore aggregates implicit and explicit gestures from consumers, and is bigger than any single company's set of data about consumers.