I'm a local LLM solutions developer and I've recently had the opportunity to spend an hour talking to the head of AI technology for one of the smaller European governments. His remit is to promote AI within the country's business community and champion local AI research and projects and so on.
We connected on a technical level as he's an older guy (as am I) and we have similar technical backgrounds and worked in similar global IT organisations. He grilled me on the AI products I'm developing for clients and went quite deeply into the queries so he is obviously much more knowledgeable than just a government official. This is his first government appointment and is very experienced in the tech industry.
But what struck me was his lack of awareness of local AI. Yes, he understood that people can d/l LLMs and run them but he had no awareness of why someone or a business would want to do this. When I explained issues of data sovereignty, he countered with ‘Copilot data protection agreements’. When I explained that legal firms are building their own local AI stacks because they've read the big AI tech agreements and don't like them and are therefore securing their own data via local LLM solutions.
We also talked about API cost risk. If a business builds AI stacks into their business reliant on API calls to OpenAI/Anthropic etc then they've created a business risk as those companies can raise API costs dramatically and business are stuck. Not to mention how frontier model companies are constantly changing their model access due to whatever internal issues of usage load or model changes and more so there's no consistency - send the same prompt via API twice and you'll like get two different answer - which is a business concern.
He also seemingly had no awareness of the backlash against big AI tech companies, how many organisations don't want to do business with companies with different values and politics as them, not to mention the green issues. I explained how local LLMs can address those issues for specific use cases to get more companies working with AI.
The conversation was good natured and he was keen to understand. But I was disappointed at how little understanding of how local LLMs can be used as an option for many business use cases. He just seems to be focused on getting businesses to send API calls to the big US AI firms. And he kept mentioning Copilot which made me cringe.
I think we, as local LLM users, need to promote local LLMs as serious business solutions for specific use cases. If we can get AI leaders to start mentioning local LLMs as a possible solution, we can perhaps gain more investment in this solution stack as a viable alternative to big AI.
Are any of you speaking to senior government people about local LLMs? What kind of conversations are you having?
[link] [comments]
