Making machines better at telling the same story in different styles

"The most durable thing in writing is style."
(Raymond Chandler)

Style is the packaging in which writing is presented, often influencing the reader's impression of the text's content. Computer scientists working in the field of text style transfer seek to develop machine learning tools to unwrap a text's packaging and repackage the same content in a different style.

One key challenge in text style transfer is finding parallel datasets—enough examples of writing where the same information is written in two different styles—for models to learn from. A new paper co-authored by Ruibo Liu, Guarini '23, and Assistant Professor Soroush Vosoughi outlines a new method that removes this need. Their novel text style transfer framework, LaMer, will be presented at the International Conference on Learning Representations 2022.

"From a technical point of view, it's interesting to see whether we can separate the content of a message from the style. It allows us to understand the different factors involved in style versus semantics in text," says Vosoughi. He draws parallels with similar methods in computer vision which take, for example, a painting by Van Gogh, and transfer the style to make it look like it was created by Picasso.

The ability to cast text into the desired target style has many applications, the authors say. It can be used for detecting plagiarism, since authors often have distinct and discernable styles. Anonymization is another—the development of tools that will help writers mask their identity. This will help people protect their identities while voicing contrarian ideas or opinions. Style transfer can also be used to align machine generated text with human values—altering toxic or biased styles are some examples. Finally, these tools can be used to create content that appeals to diverse—or even ideologically polarized—readers.

In this paper, the researchers use their model for what they call political stance transfer. News consumers may sometimes want to read views from across the political aisle, says Liu. But very often, they find the style of communication unpalatable. Such tools can convert the writer's style to one the reader is used to, allowing them to read opposing opinions while not being bothered by how they are expressed, he adds.

According to Liu, previous models approached text style transfer much like machine language translation. Unlike the one-to-one correspondence between parallel datasets used in machine translation, the datasets for text style transfer are typically not aligned. The problem is that parallel datasets of texts written from different political ideologies are not readily available and are expensive to gather manually. To overcome this hurdle, the researchers gathered content that talked about the same overall topics, but with different political stances. They then developed a framework that begins by aligning sentences from the source and target styles. Then, as part of its learning process, it mines the data to extract correspondences, refining the dataset into one it can effectively learn from. In a final step, the model employs what is called reinforced imitation learning, where the best examples of transfer are rewarded, and the worst ones penalized.

"Since the method removes the need for parallel datasets, it can be applied to any kind of text style transfer," says Vosoughi. Besides political stance transfer, the researchers also test their model on tasks like sentiment and formality transfer. They show that their novel framework matches—and in some cases outperforms—existing ones in transfer accuracy, content preservation, and fluency. Human evaluations by independent annotators hired through the crowdsourcing website Amazon Mechanical Turk show that the model also generates more readable and diverse expressions than previous models.

"A tool like ours can be used to study the way people respond to different styles of writing," says Vosoughi. "Such a tool can help people to reach communities that are not usually within their reach because of the way they write."