The rapid surge in AI investment across industries is reshaping how organisations harness the technology. With funding pouring in from both internal budgets and external sources, such as the federal government’s recent $500 billion investment to build AI infrastructure in the private sector, organisations are under increasing pressure to maximise the impact of these resources.
We at Phison believe the most significant impact will come from efforts in three main areas:
- Increasing access to AI regardless of budget
- Balancing infrastructure costs and performance to match organisational needs
- Enhancing security frameworks
Making AI accessible to all organisations large and small
A fundamental consideration for organisations receiving AI funding is how these funds are allocated. The first step for any organisation should be to make their internal knowledge easily accessible. For instance, instead of having all questions funnelled through one person, a company can train an AI that can provide references on HR policies, or customer service cases to empower individual team members.
Access to large language models (LLMs) and AI model training is currently limited by the high costs associated with computational infrastructure. Many smaller companies struggle to afford the necessary hardware to train and fine-tune AI models on-premises. This limitation forces them to rely on third-party Cloud providers, which can expose their data to security risks and reduce their control over AI-driven business processes. It also forces organisations to move data from private data centres or Edge computing locations – where data is generated or collected – to the public Cloud to be processed. This is expensive and slows response time for decision-making.
Making AI model and LLM training more accessible to organisations of all sizes is critical to fostering innovation across industries and to driving cutting-edge advancement in real-world AI applications. By increasing access to cost-effective on-premises open-source solutions, organisations can build and deploy AI models without compromising sensitive data. Lowering the barriers to entry enables a wider range of businesses to benefit from AI advancements.
But why bother training at all? The need comes about because models are trained on common datasets taken from the Internet, though they rarely include knowledge of specialist fields and certainly don’t contain internal company data. Fine-tuning models on your data improves inference capabilities – allowing AI to provide more accurate, reliable, and context-aware responses – and is essential for increasing trust in AI-driven decision-making. Another common application is to add regional languages to existing models. Many models include English, Chinese, French, Italian, German, and Spanish. Though Portuguese is close to Spanish, it’s not a perfect match. By focusing on these practical applications, organisations can ensure that AI investments lead to tangible, long-term value.
Infrastructure cost management: the role of compute and memory
As AI adoption continues to grow, so does the debate around the economics of AI infrastructure. Many organisations are grappling with how to remain competitive in an era where reduced-cost, open-source LLMs are becoming increasingly prevalent. To navigate this landscape, companies must strike a balance between performance and cost.
While compute power is often regarded as the ‘currency of AI’, memory has emerged as an equal or even more critical resource. High-performance AI models require vast amounts of memory to support training and inference tasks efficiently. The problem comes when accounting for physical board space, which limits the high-bandwidth memory (HBM) that can physically fit on a device. To get around this, companies will add more GPU to get more HBM. This can easily increase the cost of the GPU pool by 10 times when four GPU would have been enough to meet the performance target. But GPU cards are incredibly expensive and can quickly become unaffordable. This in turn forces the company to wait, sitting on the sidelines of the AI economy.
But what if there was another way to solve the problem? By adopting a more flexible approach to AI infrastructure that utilises innovative memory technology, businesses can lower on-premises solution costs, making it more feasible to deploy larger models with improved accuracy.
Security and compliance in AI deployment
Despite the advancements in AI capabilities, organisations, governments, and regulated industries face a persistent challenge: securing AI deployments and confidential data. Data injection attacks, unauthorised access, and unintentional exposure of sensitive information can undermine the credibility of AI training and inference processes.
If the cost for training AI models on-premises is out of budget, an organisation is often forced to use the Cloud. But once data leaves the boundaries of the organisation, it no longer has full control over that data.
Ideally, companies could keep their data and AI training on-premises in a closed-loop system, which allows them to train AI models on their own hardware and then deploy it to other machines, such as those in remote locations or the Edge where Internet connectivity might be limited. The data never leaves the company’s control, and the possibility of accidental leaks or data injections are greatly reduced.
The economic and technological value of a closed-loop AI solution
A closed-loop AI solution puts the security in your hands through:
- Data isolation: ensuring that AI training data remains confined within secure environments, reducing exposure to external threats
- Regulated AI layering: prohibiting uncontrolled layering from external Cloud AI training solutions to maintain consistency and security
- Compliance enforcement: aligning data usage with industry regulations and best practices to maintain legal and ethical compliance
By prioritising these security measures, organisations can build more resilient AI systems that protect data integrity and prevent exploitation by malicious actors. This, in turn, fosters greater trust in AI-driven solutions, particularly in sectors such as healthcare, finance, and government services where stringent security requirements are non-negotiable.
The value of a closed-loop system doesn’t stop at enhanced security, however. With the right technology, it can also significantly reduce the costs of AI model training and allow companies with smaller budgets to capitalise on AI technology.
Phison can help create a sustainable AI future
The rapid expansion of AI investment presents both opportunities and challenges for organisations. While increased funding can drive technological advancements, it is crucial for companies to use these resources strategically.
Thanks to its longtime commitment to cutting-edge R&D, Phison has developed aiDAPTIV, an AI model training and inference solution that drastically reduces the cost of on-premises AI usage. aiDAPTIV is a GPU memory extension solution that leverages the NAND economy of scale. By balancing cost and performance, it eliminates the need for organisations to put their data in the Cloud to train AI models – which also eliminates the associated costs of data migration – and because it allows companies to keep their data and AI training on-premises, they have more control over privacy, security, and compliance capabilities.
A well-balanced approach that prioritises cost, innovation, and security will not only ensure AI’s continued growth but also establish a foundation for ethical and responsible AI development.
About the author:
Sebastien Jean, CTO, Phison US
