AI security is coming: How can Gen-AI apps be protected against cybercrime?

The term "artificial intelligence" is not as new as we sometimes think. The term "artificial intelligence" was coined and documented by John McCarthy back in 1956. The possible applications of the early systems were very limited. Achievements such as the "Deep Thought" chess computer from the 1980s will retain their place in the history books.

The 2000s paved the way for machine learning thanks to the availability of large amounts of data. We remember: floppy discs could store a maximum of 1200 kilobytes. With DVDs, hard disk technologies and ultimately cloudisation, there were no longer any limits to data volumes. "Deep learning" was the next decisive step. ChatGPT was the next milestone and made the application of AI available to the masses. GPT stands for Generative Pre-trained Transformers. This model suddenly understood our language, could write code and had a deep understanding of context and topics. Gen-AI was thus on the rise.

Market development:

The market for artificial intelligence in Germany is still in its infancy. Nevertheless, there is already a market volume of €5 billion in these areas. This figure is expected to increase six-fold over the next five years. Growth in the field of generative AI is particularly strong in companies. The focus is on process optimisation both internally and externally

According to a McKinsey study, the proportion of companies using Generative AI has already doubled in the last three years. Companies have now firmly anchored Gen-AI in their budgets and invest an average of more than 5% of their IT budgets in development. The value potential is high, as the technology enables efficiency increases and cost reductions, among other things.

Where Gen-AI applications are used productively, I increase my attack surface. Are the consequences known?

Scalable optimisations are so exciting for companies because they have a direct impact on profitability. Automating counselling sessions, for example, saves on personnel costs, guarantees consistent quality and ensures deals are closed around the clock. However, wherever Gen-AI applications are used, companies increase their digital attack surface. For example, these applications can interact directly with customers and their input can be difficult to predict.

Example Chevrolet:

A Chevrolet dealer in the USA had introduced a chatbot to assist with various questions. A software developer used his knowledge to trick the chatbot. He recognised that the bot was based on ChatGPT and used various questioning techniques and got the chatbot to solve mathematical equations. He shared this in networks, which brought imitators to the platform. One user then managed to get the ChatBot to offer him a Chevrolet Tahoe worth around $60,000 for $1. The ChatBot was then taken offline.

But it's not just correspondence that is vulnerable. We are increasingly talking to systems or having our voice authenticated by systems. Nowadays, manipulating this voice is no longer a major hurdle.

Germany not a pioneer in Gen-AI, but with great potential

In Germany, SMEs in particular are still quite cautious when it comes to development, as issues such as compliance and regulation are not yet clear. There are currently regulations per country, but in future there are to be EU-wide resolutions to which the member states are to adhere. These processes are hampering development. In principle, however, Germany is by no means a country that is closing itself off to developments.

How can I mitigate risks?

With AI security. When developing Gen-AI applications, the security aspect should already be considered and thought through. Here are a few very popular buzzwords:

Security by design: we have long wanted this for applications beyond Gen-AI. However, it is still not consistently taken into account in the development of software and the like.
Best of all, testing: Regular tests and audits are absolutely mandatory. There are now Red Teams that specialise in uncovering the weak points of Gen-AI applications. On this basis, terms or word groups can then be selected as trigger points that restrict or terminate further communication. Companies should continuously simulate new attack scenarios and test AI models for manipulation or susceptibility to errors.
Continuous monitoring: Even after an AI system has been rolled out, it should be monitored regularly in order to recognise suspicious activities at an early stage. Artificial intelligence is capable of learning, but this also means that attackers often find new methods to trick systems.

Sources: McKinsey: "The economic potential of generative AI: The next productivity frontier" from 2023, Deloitte: Gen AI Market Outlook 2023, Statista Market Insights, Focus Online: www.focus.de/panorama/welt/softwareentwickler-manipuliert-system-auto-fuer-1-dollar-gekauft-chatbot-von-haendler-geraet-ausser-kontrolle_id_259523736.html 

We spoke to a specialist provider on these topics in our new podcast episode of Cybersecurity Basement. One of the founders explained to us what vulnerabilities he often sees and gives us a wonderful overview of AI security worldwide.

Very exciting and highly recommended.

AI security is coming: how can Gen-AI apps be protected against cybercrime?