It’s great that media queries allow us to adapt the layout of our sites to the users’ viewport size, a.k.a. Responsive Web Design. However, some argue that doing so comes with a (slight) risk of confusing some people that don’t expect viewport width to make the layout change as drastically as media queries allow.
While I don’t think this affects a large number of people, it’s quite possible that people who do not know what responsive web design is (i.e. the vast majority of people) can be confused by a site looking radically different on their computer that on their tablet or smartphone. Bruce Lawson mentions an example of this in Turning off responsive web design, and I’ve heard other similar stories.