Why AI Might Not Make the Best Coding Buddy

Author: Marcel den Hartog, CGEIT, Trend & Innovations Expert at Enable U in The Netherlands
Date Published: 15 March 2024

It is so easy: “AI (Bard, ChatGPT, Claude, Copilot), generate me some Java code that does ABC, please check this code for inconsistencies, generate a price quote based on the following information, please generate an extract of the following 40-page report.”

And, presto, with or without fine-tuning, you get what you asked for. Because people “feel” as if they have a personal conversation with something that’s social and understanding (remember that they are all developed to function as chatbots), there are hardly any barriers to what people will ask generative AI. People often forget the internet adagio “you pay with your information.” So, I did a test.

Test 1: AI uses feedback instantly

I logged on as user A and asked both Bard and ChatGPT what the target verticals for my company are. And the answer I got was straight from our website, Government, Education and Finance. I then asked what our strategy should be if we would add healthcare as an extra vertical. During the dialogue that followed, both AI solutions created a decent go-to-market strategy for us.

I then asked a colleague to log on and repeat the question I asked initially. Suddenly, healthcare was not just marked as an extra interesting vertical, it was mentioned as our #1 target vertical! Bard used the information I provided earlier instantly to generate a response for another user.

Test 2: AI has no knowledge of the term “Confidential”

Government strategy papers tend to be long and boring. I suspected that many of these reports are uploaded to create readable, four-page summaries. I asked Bard, ChatGPT and Copilot the following question: Please give me in-depth information about the IT strategy for the next five years for XXXXXX (a large city in The Netherlands).

After trying four different names of large cities, the feedback from Bard was a lengthy answer with the name of a report and many details of the IT strategy. It even gave me quotes directly from the report. City XXXXX happens to be a particularly good client of ours, but we were not familiar with the report. The report cannot be found on their website and even with a deep Google search I could not find any reference to the report. The report with the stamp “Confidential” on it was simply uploaded during a “personal” dialog between a user and an AI application.

 Key Takeaways

  • Most people are not aware that the information they enter into public AI applications, like ChatGPT, Bard and others, is public domain as soon as they press the enter key. “When the product or service is free, you pay with your information.”
  • AI is loudly hyped so everybody will try and use it, sometimes with amazing results. It will not be easy to make organizations aware of the risks involved in using public AI tools.
  • Storytelling and down-to-earth examples are the only way to reach leaders, managers and employees when warning about risks of using such a popular tool. “If everybody is using it, why should we put any restrictions on it?!” So, ask questions like:
    • Are you willing to give this information to any stranger you see in the departure hall of an airport? If the answer is “yes,” it is ok to feed it to AI.
    • If you want to optimize a code snippet that is part of your IP stack, do not give it to AI.
    • If your five-year strategy plan can be read aloud near the reception of your top competitor’s headquarters, it is fine to ask AI to create a four-page summary.
    • If you ask AI to create a price offer, the pricing, warranty and other essential information you upload can be retrieved by others easily.
    • Is the information HR uses to create a job-description for a C-Level manager confidential? If so, do not upload into a public AI solution.
    • If the instructions for the code you want AI to generate describe what your product will do, you may want to rethink if you want to use AI as your coding buddy.
    • And the list goes on.

The two tests I did are simple, and I am sure you can think of variations that will feel relevant to your organization. But remember, when you warn people about some of the effects of immensely popular modern technology, they might feel restricted and attacked. AI has entered the world with a big bang and has many beneficial uses. We must, however, make sure we use it without losing control of the confidential and valuable data that make our company unique and successful.

Additional resources