Zoho has announced plans to use the NVIDIA accelerated computing platform, including NVIDIA NeMo. The company plans to leverage the technology to build large language models (LLMs) for its Software-as-a-Service (SaaS) applications.
Over the last year, Zoho has spent over $10 million on NVIDIA AI technology and GPUs. This announcement takes that investment further, although it does not value it. NVIDIA’s NeMo is an end-to-end platform for building custom generative AI models.
While not specifically mentioned in the announcement, Zoho likely plans to deploy NVIDIA NeMo inside its data centres. That would enable it to build custom models for its customers at scale and deploy where they are needed. Additionally, such a move removes latency, enabling it to expand its offerings.
Ramprakash Ramamoorthy, Director of AI at Zoho Corporation, commented, “Many LLMs on the market today are designed for consumer use, offering limited value for businesses. At Zoho, our mission is to develop LLMs tailored specifically for a wide range of business use cases.
“Owning our entire tech stack, with products spanning various business functions, allows us to integrate the essential element that makes AI truly effective: context.”
Deepening investment in AI to meet customer demand
Customers are demanding more AI solutions from vendors, and they want those that are customised rather than generic. There are three main reasons for that. The first is that they want AI to give them a business advantage. This will not happen when using a generic model that is available to everyone.
Secondly, they want accuracy and trust. The hallucinations and lack of transparency in public models cause problems for organisations, especially around PII. They cannot show where the data came from, and worse, they cannot delete the data. It is becoming a serious problem as regulators pay closer attention to AI.
Lastly, a custom GenAI model built on an organisations data allows it to get significant insight into its own data. The answers can be quickly aligned with current business plans and used to create new strategies. More importantly, it creates a trust in the data that reduces any regulatory risk from data misuse.
All of these play to Zoho’s existing AI strategy. Its approach to building private LLMs for customers has been key to its success in this area. It has also ensured that its own AI tools inside its SaaS solutions integrate and utilise those models effectively. This can only happen when you own the entire AI lifecycle.
A new breed of models
There is another advantage to Zoho and therefore to its customers of having the technology inside its data centres. Customers are moving away from a single LLM encompassing the entire business to Small and Medium Language Models (SML, MML). These are much more focused on a single business function or a business unit.
Importantly, it means that the maintenance and tuning of those models is extremely fast compared to LLMs. It also enhances privacy by preventing PII being exposed throughout the business by containing it in SMLs.
From a business perspective, this does not replicate the data silo problem that they currently struggle with. That is because the SaaS solutions are capable of asking questions of multiple SMLs and MMLs. It’s an AI fabric that sits on the data fabric many organisations have invested heavily in over the last decade.
What will Zoho do next with NVIDIA technology?
While the release did not mention any specific future projects, it did say that the company is testing NVIDIA TensorRT-LLM. It is already seeing benefits from that work with a 60% increase in throughput and a 35% reduction in latency compared with other technologies it has used.
Another project that Zoho is working on is speech-to-text using the NVIDIA accelerated computing infrastructure. This is a market where AI has brought significant change over the last few years. It has rapidly improved the transcription market and made it easier to add new languages. It is also opening up the ability to ask for summaries of that audio transcription across single and even multiples recordings .
Zoom recently announced its new capabilities in this space with AI Companion 2.0. The research that Zoho is doing suggests it is planning something similar with Zoho Meeting.
Enterprise Times: What does this mean?
Getting closer to NVIDIA and delivering more focused solutions to its customers is a smart move by Zoho. Its existing privacy focused approach to AI models is paying off and this will only help it expand that.
The investment of $10 million plus this latest move in NVIDIA products over the last year is good but really just table stakes. The question is, how much more will Zoho spend to build out its AI infrastructure as it looks to compete in a fast-growing market?
Another, and more serious question is, does Zoho have the power infrastructure at all its data centres to support a wider deployment of AI technology? Many data centres are discovering that the power demands of GPU driven data centres are hard to meet in their current location. It has led to a new phase of data centre building, just for AI. It will be interesting, therefore, to see if Zoho announces any AI-specific data centre builds.
