It’s 1885, and an English civil servant walks down the gangplank of a boat and into the teeming markets of Kolkata. He has only recently arrived in British-held India, and he is unprepared for the heavy humidity of the air, the merciless sun. He wasn’t expecting the riot of smells: fruit and pollen, offal and sewage, incense and livestock. The market is a kaleidoscope of color, hurting his eyes. The crowd is speaking in a language he does not understand, their voices loud, their rough feet slapping against the dust. He is overwhelmed. Being British, his discomposure must be India’s fault. After all, what else can the British Empire expect from such “uncivilized” people?

But “civilization” and “empire” are tricky subjects, says Andrew J. Rotter, Charles A. Dana Professor of history and peace and conflict studies. In his new book, Empires of the Senses, published by Oxford University Press, Rotter offers a study of the ways that English-speaking powers — the British in India, the Americans in the Philippines — experienced the landscapes they had conquered, and how their experiences colored their beliefs about those landscapes. Drawn together through archival research in four different countries, the book is a social history of two empires and the influence that the five human senses had on them.

“I’m interested in encounters on the ground in India and the Philippines,” Rotter said, noting that he looked at archival reports not only from officials, but also from more ordinary figures (including women, who often proved more descriptive than men.) “I look more or less at ordinary people who came out to these places and wrote home about them or kept diaries about them, because they’re creating a broader idea of how empire works. They’re the ones who are making the empire function.”

Both empires began with intense spasms of violence, Rotter said. Beginning in the late 18th century, a private corporation — the East India Trading Company — ruled India. They mismanaged affairs so badly that they provoked a major rebellion in 1857, which ultimately required imperial forces to revoke their charter, savagely crush the revolt, and assume official state control. In 1898, American forces snatched the Philippines from Spain with promises to help the Filipinos achieve independence: when it became clear the Americans weren’t actually going to leave, the Philippines was rocked by a revolt of its own. In both cases, thousands died. The conquerers, meanwhile, were convinced that the bloodshed was for the colonies’ own good.

The sensory experiences of British and American visitors to their respective colonies only strengthened that belief. Long before India appeared as a line over the horizon, British visitors claimed to be able to smell it. Upon arrival, they were met by some scents that were pleasant or enticing: new perfumes, fruits, and flowers. Other sensory experiences were less pleasant: the odor of human and animal waste; human bodies that had not been washed to English specifications; crowds of people they deemed unruly, with a lack of proper English manners and customs.

“Most of the people who came over had strong racial prejudices, so dark skin of the people formed a particular kind of impression of them,” Rotter said. “In terms of sound, the volume of these places, the noise they experienced, and their inability to understand what people were saying confounded them and made them fearful that they were being plotted against.”

The colonized weren’t terribly impressed with their rulers, either. Rotter found accounts by Indians who were disgusted by British habits like the handkerchief: what kind of barbarians carried snot-filled pieces of cloth around in their pockets? Hindus used incense as part of religious ritual, which the largely Protestant British abhorred; Indian nobility invited to fancy dinner parties were often utterly baffled by the profusion of silverware on display — not least of which because Indians ate with their fingers, which they considered the cleaner option.

“There are scattered examples of Indian impressions of how the British complicated things needlessly or didn’t understand the right way to do things,” Rotter said. “Who was ‘civilized’ when the British came over and began spreading venereal disease because they couldn’t or wouldn’t control themselves sexually? The Indians didn’t do that. Who among us is ‘uncivilized’ when they came to our country, first of all, in order to conquer us?”

British and American colonizers, meanwhile, considered their specific brand of civilization to be a gift, albeit a paternalistic one, that their subject peoples had no power to refuse. Weren’t they installing roads, creating more mobile societies, overhauling sewage systems, and generally reshaping the societies along sensible western lines?

“The British felt that it was impossible to get millions and millions of Indian farmers civilized, to get them to clean up, to reform themselves, to cure them of all diseases,” Rotter said. “But it might be possible to get a narrower stratum of elite Indians to do these things, to serve as a model for the rest of the people in India. And this belief in ‘civilization’ was a central part of making the Philippines and India respectively suitable for self-government.”

Yet the notion of self-government proved a bit of a problem for both imperial powers. For some parties — particularly those in America and Britain — it was the ultimate goal of colonization, as much from pragmatism as idealism. (It’s very difficult and expensive to keep hold of a territory half a world away.) Still, both empires proved reluctant to let their territorial acquisitions slip away. Whenever calls for independence bloomed, Rotter said, the response was the same: “They’re not ready. They’re not civilized. They cannot fend for themselves … we cannot in good conscience pull out.”

By 1948, both India and the Philippines had finally won their independence. But the paternalistic blend of contempt and charity in the service of imperial ambitions did not end. “The same kinds of language that were used by the Americans in the Philippines was just as quickly and easily revived and applied to people in Vietnam,” Rotter said. “If we pull out of Vietnam, they will not be able to govern themselves successfully. And the kind of language you heard people using in the aftermath of 9/11 to describe both enemies and allies in the Middle East was reminiscent, I think, of other kinds of racial epithets previously used to describe Filipinos and Vietnamese.”

The lesson here, Rotter said, is that empire and culture aren’t things that happen on a map: they’re things that happen in the body and its experiences. Without grappling with these seemingly minor details, it’s impossible to get a complete picture of history. “People who go to these places are not disembodied souls,” Rotter said. “They’re not just ciphers who are sitting in offices, managing things. They’re immersing themselves in the place to which they’ve been sent.”