Article Details

OpenAI Bans ChatGPT Accounts Used by Russian, Iranian and Chinese Hacker Groups. OpenAI has revealed that it banned a set of ChatGPT accounts that were likely operated by Russian-speaking threat actors and two Chinese nation-state hacking groups to assist with malware development, social media automation, and research about U.S. satellite communications technologies, among other things. "The [Russian-speaking] actor used our models to assist with developing and refining Windows malware, debugging code across multiple languages, and setting up their command-and-control infrastructure," OpenAI said in its threat intelligence report. "The actor demonstrated knowledge of Windows internals and exhibited some operational security behaviors." The Go-based malware campaign has been codenamed ScopeCreep by the artificial intelligence (AI) company. There is no evidence that the activity was widespread in nature. The threat actor, per OpenAI, used temporary email accounts to sign up for ChatGPT, using each of the created accounts to have one conversation to make a single incremental improvement to their malicious software. They subsequently abandoned the account and moved on to the next. This practice of using a network of accounts to fine-tune their code highlights the adversary's focus on operational security (OPSEC), OpenAI added. The attackers then distributed the AI-assisted malware through a publicly available code repository that impersonated a legitimate video game crosshair overlay tool called Crosshair X. Users who ended up downloading the trojanized version of the software had their systems infected by a malware loader that would then proceed to retrieve additional payloads from an external server and execute them. "From there, the malware was designed to initiate a multi-stage process to escalate privileges, establish stealthy persistence, notify the threat actor, and exfiltrate sensitive data while evading detection," OpenAI said. "The malware is designed to escalate privileges by relaunching with ShellExecuteW and attempts to evade detection by using PowerShell to programmatically exclude itself from Windows Defender, suppressing console windows, and inserting timing delays." Among other tactics incorporated by ScopeCreep include the use of Base64-encoding to obfuscate payloads, DLL side-loading techniques, and SOCKS5 proxies to conceal their source IP addresses. The end goal of the malware is to harvest credentials, tokens, and cookies stored in web browsers, and exfiltrate them to the attacker. It's also capable of sending alerts to a Telegram channel operated by the threat actors when new victims are compromised. OpenAI noted that the threat actor asked its models to debug a Go code snippet related to an HTTPS request, as well as sought help with integrating Telegram API and using PowerShell commands via Go to modify Windows Defender settings, specifically when it comes to adding antivirus exclusions. The second group of ChatGPT accounts disabled by OpenAI are said to be associated with two hacking groups attributed to China: ATP5 (aka Bronze Fleetwood, Keyhole Panda, Manganese, and UNC2630) and APT15 (aka Flea, Nylon Typhoon, Playful Taurus, Royal APT, and Vixen Panda) While one subset engaged with the AI chatbot on matters related to open-source research into various entities of interest and technical topics, as well as to modify scripts or troubleshooting system configurations. "Another subset of the threat actors appeared to be attempting to engage in development of support activities including Linux system administration, software development, and infrastructure setup," OpenAI said. "For these activities, the threat actors used our models to troubleshoot configurations, modify software, and perform research on implementation details." This consisted of asking for assistance building software packages for offline deployment and advice pertaining to configured firewalls and name servers. The threat actors engaged in both web and Android app development activities. In addition, the China-linked clusters weaponized ChatGPT to work on a brute-force script that can break into FTP servers, research about using large-language models (LLMs) to automate penetration testing, and develop code to manage a fleet of Android devices to programmatically post or like content on social media platforms like Facebook, Instagram, TikTok, and X. Some of the other observed malicious activity clusters that harnessed ChatGPT in nefarious ways are listed below - "Some of these companies operated by charging new recruits substantial joining fees, then using a portion of those funds to pay existing 'employees' just enough to maintain their engagement," OpenAI's Ben Nimmo, Albert Zhang, Sophia Farquhar, Max Murphy, and Kimo Bumanglag said. "This structure is characteristic of task scams."