Cybersecurity in the age of AI

We are living a historical peak moment in hype for artificial intelligence. In a matter of one year generative models have sneaked into our everyday lives and become the biggest headache for teachers correcting homework. This fast development and general usage has resulted in spreading the idea of artificial intelligence as a digital servant that thinks for us and gives us entertaining conversations. This popular concept might actually be underestimating the potential of these tools for the good will, but also for the bad. By the way, I used a free image generator in a webpage to generate the picture below, for it to work as a post miniature in the blog feed.

AI as a (new) threat

If you have an email address or a phone number, I am almost certain you have been target of some sort of scam attempt. These scams generally try to either steal your credentials for some application or convince you to send someone some amount of money for whatever reason that could make you think of it as a good idea. They either make you think it is some friend asking for money or steal the identity of your bank to tell you how your money is in danger and should transfer it somewhere "safe" (even spoofing their official phone numbers when calling you). And actually, we shouldn't feel stupid at all for falling into these tricks at some point. It is called social engineering and it targets our psychological biases and weaknesses. The more customized towards the victim social engineering is, the harder it is to detect.

In this sense, generative language models such as ChatGPT aren't discovering anything new, but they sure make it way easier and faster to prepare this sort of attacks. Good social engineering requires a realistic context, able to convince someone to do something they wouldn't normally do. Tools such as ChatGPT are able to generate good quality and convincing scams without effort, but also, and considering how fast this models are developing, to what extent would you be able to tell if you're chatting with a real person or a robot? 

This real time chatbot thing has been in the wild for some time now, implemented in several services and web pages you might have come across. But until now, the results have been poor in terms of how natural it feels to the user. This is changing, and in the very short term it will be very hard to tell robots from people in chat conversations. Have you ever seen these scams where somebody texts you pretending to be your relative and saying how urgently they need money to avoid some ugly situation? For the average person it is pretty easy to tell it is a scam. You might even think there is no real threat behind these messages and see it more as a comedic anecdote. But these scammers keep doing it for a reason: it works. It may fail nine out of ten times, but the one time it succeeds, makes it profitable. Now, can you imagine this business if attackers were able to target hundreds of people at the same time without having to text them one by one? Hundreds of parallel conversations conducted by robots that are actually able to talk even more convincingly than the foreign scammer that doesn't even speak your language. You can be sure that at some point, sooner than later, this business is going to be a thing.

We have some more recent examples of how generative AI has had cybersecurity implications. We could talk about how GitHub copilot has leaked credentials or how it can introduce insecure code into your software. This naturally has to do with how the samples of code used to train the model contained hard-coded secrets and vulnerabilities. There has also been the story of some users reporting how it was possible to have Bard or ChatGPT provide limited but usable Windows keys.

AI in threat detection

AI does one single thing but it does it extremely well: it learns patterns. That's why it works fine for generation of data (creating it from the patterns it knows) and detection (observing patterns it knows among bigger amounts of data). And if AI is good at recognizing our face in a picture, it definitely can be good at detecting attack patterns in huge amounts of log data.

If you are familiar with security monitoring you might have an idea of how a SIEM in a fairly big organization is capable of ingesting several bytes of log data everyday. If you don't know what a SIEM is, you can understand it as a system that gathers log data from different sources into the same place for it to be analyzed for security purposes. A log source can be any device connected to the network that is logging whatever happens in it (log in/log out operations, network connections, file access/creation/deletion... you name it). In a company with several hundred laptops, a VPN server, a proxy server and several other intranet services, you can imagine the vast amount of data that is being generated every second. And all that data is (should be) analyzed in search for possible threats and security events affecting your business.

Throughout the last years, machine learning ("AI") algorithms have come really handy in supporting analysts in recognizing suspicious patterns among the data. When you have so much information to analyze in search for threats, it saves a lot of precious time to drive the focus to where the attack could actually be. This has powerful applications in, for example, insider threat detection (a company's own employees trying to damage it). Insider detection involves analyzing behavioral patterns in the employees that could evidence something bad is happening. These patterns could be simple things such as an employee connecting at strange hours or abnormal amounts of attached files being sent to external email addresses, which could be a signal of important data being leaked. Imagine having to monitor all this data manually. Instead, AI algorithms, through simple statistics, are capable of quickly identifying this sort of behaviors and highlighting them to the analyst for a more in depth study of the relevant cases. This is just an example of how AI can serve the purpose of efficiently monitoring an organization for cybersecurity threats.

From the cybersecurity point of view, it is clear AI is gaining relevance and there is a lot more to come in the short term. As it has happened with every technological advance or revolution, there are and will be unforeseen implications. Not being agile enough in managing those implications will open the door to considerably negative consequences. There will be more to talk about in this matters.

Thank you for reading. Feel free to share your thoughts in the comments.