Priyanka Ranade is a Ph.D. student at the University of Maryland, Baltimore County, studying computer science and electrical engineering. Anupam Joshi is a professor at the University of Maryland, Baltimore County, where he teaches computer science and electrical engineering. Tim Finin is a professor at the University of Maryland, Baltimore County, where he teaches computer science and electrical engineering. This article first appeared on The Conversation.
You may have come across posts flagged with disinformation warnings if you use social media platforms like Facebook and Twitter. The majority of misinformation, both marked and unflagged, has thus far been directed at the general population. Consider the potential for misinformation—false or misleading information—in scientific and technical disciplines such as cybersecurity, public safety, and medicine.
Misinformation is spreading in these crucial sectors as a result of widespread biases and practices in publishing scientific material, including in peer-reviewed research publications, which is causing significant worry. We explored a new pathway of misinformation in the scientific community as graduate students and faculty members performing cybersecurity research. We discovered that artificial intelligence systems may generate fake information persuasive enough to dupe specialists in crucial sectors like medicine and defense.
Misinformation aimed at tarnishing the reputation of companies or public personalities is common. Misinformation in expert groups has the potential to have frightening consequences, such as doctors and patients receiving wrong medical advice. This could endanger people’s lives.
We investigated the effects of propagating disinformation in the cybersecurity and medical communities to test this issue. We created phony cybersecurity news and COVID-19 medical research using artificial intelligence models called transformers and delivered the cybersecurity disinformation to cybersecurity professionals for testing. We discovered that misinformation generated by transformers was able to deceive cybersecurity specialists.
Artificial intelligence powers most of the technology used to detect and handle misinformation. Because there is too much misinformation for individuals to discover without the aid of technology, AI allows computer scientists to quickly fact-check vast amounts of it. Although AI aids in the detection of false information, it has also been used to spread false information in recent years.
Natural language processing is used by transformers like Google’s BERT and OpenAI’s GPT to understand the text and produce translations, summaries, and interpretations. They’ve been used in tasks like storytelling and answering questions, pushing the limits of machines with human-like text generation abilities.
Transformers have aided Google and other technology companies in improving their search engines, as well as assisting the general public in overcoming common issues such as writer’s block.
Transformers can be utilized for nefarious reasons as well. Artificial intelligence-generated fake news has already been a problem for social media sites like Facebook and Twitter.
Important erroneous information
Transformers, according to our research, also offer a disinformation hazard in medical and cybersecurity. We fine-tuned the GPT-2 transformer model on available Internet sources detailing cybersecurity vulnerabilities and attack information to demonstrate how dangerous this is. A cybersecurity vulnerability is a computer system’s flaw, and a cybersecurity assault is an act that takes advantage of that flaw. For example, if a vulnerability is a weak Facebook password, a hacker figuring out your password and breaking into your account would constitute an attack exploiting it.
The model was then fed the rest of the threat description after being seeded with a line or phrase from an actual cyber threat intelligence sample. This created description was submitted to cyberthreat hunters, who sift through a lot of data on cybersecurity dangers. These experts analyze threat descriptions to spot prospective attacks and alter their systems’ defenses.
We were taken aback by the results. Cyberthreat hunters, who are well-versed in all types of cybersecurity threats and vulnerabilities, were duped by the cybersecurity disinformation examples we created. Consider this scenario with a critical piece of cyberthreat intelligence involving the aviation industry that we gathered as part of our research.
This deceptive piece of information offers inaccurate information about cyberattacks on airlines that handle sensitive real-time flight data. By diverting their attention to bogus software defects, this bogus information may prevent cyber analysts from resolving actual vulnerabilities in their systems. In a real-world scenario, a cyber analyst acting on the phony information may have exposed the airline to a serious attack that exploited a real, unresolved vulnerability.
In the medical domain, a comparable transformer-based approach can create information and potentially deceive medical specialists. Preprints of scientific publications that have not yet been peer-reviewed are constantly being uploaded to sites like medrXiv during the COVID-19 epidemic. They are being utilized to make public health choices as well as being described in the press. Consider the following, which is not genuine but was created by our model on certain COVID-19-related publications after minor fine-tuning of the default GPT-2.
The program was able to create whole phrases and an abstract that purportedly described the negative effects of COVID-19 vaccines as well as the tests that were carried out. This is concerning for medical researchers, who rely on reliable data to make informed judgments, as well as members of the general public, who frequently rely on public news to learn about important health information. If believed to be true, this type of misinformation could endanger lives by diverting scientists’ efforts in biological research.
Is there a misinformation arms race between AI and humans?
Even though cases like this from our study can be verified, transformer-generated misinformation prevents businesses like health care and cybersecurity from employing AI to aid with information overload. Automated systems, for example, are being created to extract data from cyberthreat intelligence, which will subsequently be used to instruct and train automated systems to spot potential assaults. These automated systems will be less effective at detecting actual threats if they evaluate such fake cybersecurity text.
We believe that as persons distributing disinformation find better techniques to manufacture false information in response to effective techniques to recognize it, the result might be an arms race.
Researchers in the field of cybersecurity is always looking for new approaches to detect misinformation in many domains. Understanding how misinformation is generated naturally aids in understanding how to spot it. Automatically created content, for example, frequently contains tiny grammatical errors that systems can be trained to detect. Systems can also cross-correlate data from numerous sources to discover statements that aren’t backed up by additional evidence.
Finally, everyone should be more conscious of what information is reliable and be aware that hackers prey on people’s confidence, particularly if the material is not sourced from respected news sources or published scientific studies.