By far and and away the most common criticism of utilitarianism can be reduced simply to: "I don't like it" or "It doesn't suit my way of thinking". For an example of this, here's something from someone who might prefer to remain nameless.
"Producing the greatest good for the greatest number is fine as long as you are not hurting someone you really love in the process. For instance, with the trolley situation, I would rather kill 5 people on the main track than m mother on the spur track. Utilitarianism runs into problems when sentiment is involved!!"
I suggest one certainly will have a problem if one tries to merely codify one's personal inclinations - and then expects this to hold as a universal standard of right and wrong - however, the problem is not with utility!
Utilitarianism is alleged to be faulty in the way it requires us to think about all kinds of actions - to apply the felicific calculus in disregard to any feared distaste of the result. For example, some issues or potential actions are (to a non-utilitarian) "morally unthinkable":
"Consequentalist rationality, however, and in particular utilitarian rationality, has no such limitations: making the best of a bad job is one of its maxims, and it will have something to say even on the difference between massacring seven million, and massacring seven million and one." 
Utilitarianism does indeed have something to say on this issue - otherwise it would suggest that the life of this extra individual was of no importance. I suggest it as a virtue of utility, that it does not arbitrarily discount value depending on some detail of the situation: all interests count - simply and fairly. The fact that opponents of utilitarianism admit that they won't even consider some situations seems to me to be most damning to their credibility, and indicative of their general irrationality on matters ethical.
The argument from distaste is often expressed as a suggestion that utilitarianism doesn't provide enough support for individuals' rights. But what is a right, and what is its justification? If the justification of a right depends on its tendency to promote happiness and prevent suffering, then it is entirely redundant since this is the sole purpose of utility. And if rights aren't justified in these terms, how are they justified - what on earth are they actually good for? Of what use are they?
It is generally found that the proponent of ethical rights has very unclear thinking as to what rights are and why they (should) exist - and it is therefore of unclear importance that utilitarianism does not support them.
Doesn't utilitarianism imply that, if we found a drug which had the sole effect of producing happiness, we ought to mass produce and consume it? And, since happiness is just an emotion which can be chemically induced, isn't it a bit silly to make it the highest order objective? 
It is quite strange that many people will accept "the pursuit of happiness" as one of life's fundamental entitlements, yet should suddenly develop ascetic inclinations as soon as the quarry appears obtainable. It seems they don't have a problem with someone trying to achieve happiness, rather they are only concerned when that someone has a reasonable prospect of success in their attempts. Perhaps their fixation with unhappiness would be satisfied by personally abstaining from joy - but, if it goes further such that they would attempt to prevent individuals from attaining happiness even at no cost to others, then (from a utilitarian point of view) such people are despotical and a menace to society.
It is possible that many people's aversion to the idea of everlasting happiness is caused by incomplete consideration of the issue. It could be that people have become so jaded by mistaken claims for the desirability of various intentional objects that they believe that drug-induced happiness simply would not be durably satisfying. Since any notion of happiness worthy of the name includes that of satisfaction, it follows that a truly happy person cannot be dissatisfied, so this problem can never arise.
Happiness, in the utilitarian sense, includes the exemption from suffering. A charge of triviality for pleasure can perhaps be made, if our only frame of reference is the knowledge of felicific states currently achievable, but it is altogether less plausible against the depths of suffering currently experienced by the world's less fortunate beings.
The second most common criticism of utilitarianism is that it is impossible to apply - that happiness (etc) cannot be quantified or measured, that there is no way of calculating a trade-off between intensity and extent, or intensity and probability (etc), or comparing happiness to suffering.
If happiness was not measurable, words like "happier" or "happiest" could have no meaning: "I was happier yesterday than I am today" would make no sense at all - it can only have the meaning which we (or most of us, at any rate) know that it has if we assume that happiness can be measured and compared.
"one should face the fact that goods are not necessarily intersubstitutable and consider the case, for instance, of an intransigent landowner who, when his avenue of limes is to be destroyed for the motorway, asks for 1p compensation, since nothing can be compensation." 
(One is reminded of the story of the mother handing out home-baked cookies as a special treat to her family. The youngest child, on finding his cookie to be slightly smaller than the others, smashes it up and storms out in tears. In his disappointment, he interprets a fine gift as an affront, and he would rather make things worse than better - but then he's only a child. Adults, of course, have much less obvious and more subtle means of smashing their cookies.)
Initially, it seems very odd that the landowner should ask for a penny. If nothing can be compensation, why does he not ask for nothing? What use is this tiny amount of money? Far from suggesting that the trees are invaluable, it suggests that any money he could get for them is worthless to him! But, we may still ask, why the penny? And then we realize: it's a token; a chip in a psychological game (often called "Poor me!"). One can imagine the penny being carried about by the ex-landowner, and produced to evict pity from those unfortunates he manages to convince to listen to his story. That will be his best effort at compensating himself.
Now suppose the scenario is amended slightly: imagine the landowner's daughter is dying from a terminal disease; that the motorway's supporters offer to pay for the new and expensive cure (which the landowner could not otherwise afford) in exchange for the land; and that they will not proceed without his permission. Are we still to presume that "nothing can be compensation" for his trees, not even the life of his daughter? Or will the landowner decide that his daughter's life is more important than his pretty view? It seems likely.
But suppose not - suppose he chooses to keep the trees and lose his daughter. Does this show that the value of the lime avenue isn't convertible? Of course not, just that he values the trees more than his offspring. If the two different values were inconvertible, he would have no way to decide one way or the other - no way to choose between them. The fact that people can and do weigh-up and trade-off values, for all types of things, shows that it is both possible and practical to do so.
In the original scenario, the sensible thing to do would be to ask for enough money to buy a new bit of land, and to plant a new avenue of limes on it; but, since the principle of utility does not imply the absence of fools, this criticism has no effect, and we needn't consider this matter further.
The third most common criticism is that it is too difficult to apply - that we cannot calculate all the effects for all the individuals (either because of the large number of individuals involved, and/or because of the uncertainty). The principle of utility is, essentially, a description of what makes something right or wrong - so in order for it to fail, someone must give an example of something which is useful but obviously wrong. The principle does not imply that we can calculate what is right or wrong - completely accurately, in advance, or at all! It does not harm the principle of utility at all merely to comment that it is difficult for us to work out what is right - it is merely a lament against the human condition.
The idea of practicality is often used to suggest a problem exists in the theory, when it fact it does not. For example:
"how far does one, under utilitarianism, have to research into the possibilities of maximally beneficent action, including prevention?" 
The answer is simple, and entirely obvious: as far as it is useful to do so! That is, far enough so that we get the optimal trade-off between planning and implementing, so that we maximize our effectiveness as agents. The does imply that, in some cases, it may not be best to apply the felicific calculus at all: if the problem is one that we have faced many times before, and always reached the same conclusion; or if the case presents itself as an emergency, and isn't open to extended consideration; we can forego the calculus and act immediately.
One argument which some people propose as being more sensible than other criticisms, is that utilitarianism is "fine, so far as it goes", but that it fails to consider some sources of value, and that it will therefore produce the wrong results when these different sources conflict. There is potential for confusion here - sometimes "utilitarianism" is used to specifically for "hedonistic utilitarianism"; and, sometimes, it means a particular class of ethical theory (something like "value-maximizing consequentialism") ... under this meaning, an ethical theory which held the existence of plastic forks as supremely valuable, and therefore tried to maximize their number, would be "plastic fork utilitarianism".  So, theories which have other intrinsic values than happiness and exemption from suffering can be accommodated within a utilitarian scheme.
As for those other things that are suggested as having value, there are a few worth mentioning: "life", "friendship", and "knowledge" among them. I think it is notable that these things are valued, but that they also generally create happiness... I suggest the reason that they are valued is precisely because they promote happiness. But, if they didn't, would we still value them? Does someone who suffers too much still value their life? Surely not, or else there would be no suicides. Do we value a friendship if we get no pleasure from it? On the contrary, it is more likely that we would define our friends as those people about whom we enjoyed being. And is it worthwhile learning and philosophising, if our knowledge is never of any use at all? Or, rather, is it just so much meta-physical stamp collecting? The case against these "other" goals is quite clear.
A reply to a particular criticism by Tom Regan in "The Struggle for Animal Rights" is available here. Answers to his earlier criticisms in "The Case For Animal Rights" are available here.
One: that it opposes happiness to usefulness.
Two: that it is only concerned with the greatest happiness of the greatest number of people - unfortunately this mistake has even been made by some who call themselves utilitarians. From the very first formulation by Bentham, to the more recent modifications by Singer, the principle of Utility has concerned itself with all happiness, no matter the identity of the being in which it is felt. The interests of non-human animals must be counted equally with those of humans; and if we are ever visited by UFOs, we must consider the alien's interests too.
"We are perfectly willing to stake the whole question on this one issue. Granted that any practice causes more pain to animals than it gives pleasure to man; is that practice moral or immoral? And if, exactly in proportion as human beings raise their heads out of the slough of selfishness, they do not with one voice answer 'immoral', let the morality of the principle of utility be forever condemned."
"the demands of political reality and the complexities of political thought are obstinately what they are, and in the face of them the simple-mindedness of utilitarianism disqualifies it totally. The important issues that utilitarianism raises should be discussed in contexts more rewarding than that of utilitarianism itself. The day cannot be too far off in which we hear no more of it."
utilitarian.org has been laughing in the face of Bernard Williams (and other non-consequentalists) since 18th October 1999.