That entirely depends on the circumstances. One cannot make a blanket statement. Let’s say that a horrible disease breaks out in a city and the government successfullymanages to quarantine the city before anyone gets out. The people inside suffer and most die without medical help or medicine, but the disease doesn’t spread and humanity is saved.

However there are plenty of cases where we should stand up for the rights of a few people even if it costs the majority of people. Making things accessible for people with disabilities costs money that technically could be spent elsewhere, but an empathic society should stand above this kind of thinking and make sure that society is accessible to people with disabilities.