Editor's note: Heading towards 2025 Corporate Board Members We are launching a new “Ask Me Anything” style column with Karen Silverman and Joann Stonier of Cantellus Group, global experts in practical governance strategies for AI and other technologies. First, we're digging into the troublesome question: whether we'll invest in building our own AI models based on our own data or rely on public models developed by companies like Openai, Meta, Google and more. This is what they had to say:
It's a very important and tough question and it depends heavily on what you're at risk. Perhaps we will deploy both solutions over time, but a clear assessment of our business models, resources and capabilities can help us decide where to start.
Public LLMS They are all trained with the same (mostly western) world data. The LLMs from major AI Labs are well designed and always improve power, accuracy and capabilities. These are good starting points for many tasks, including summarizing large quantities of non-sensitive material, generating drafts, brainstorming different perspectives and general options.
Considerations include the need for substantial monitoring and user training, even for repetitive tasks and protection of your own data when interacting with the API. Lack of competitive differentiation. Reliance on training data that may be subject to litigation. Lack of control over feature development (everyone gets the same feature at the same time). Lack of explanation, bias, drift, and potential loss of performance. Lack of accuracy and challenges with reliability, accuracy and security.
Unique LLMSIn contrast, it is developed in-house, tailored to your specific needs and trained with unique data. The results are ideal for highly differentiated tasks when company-specific texts, facts or practices, or security, reliability, and/or regulatory requirements are at premium.
Considerations here include non-trivial costs (in time, talent, treasure) for development, testing and verification. Access to sufficient volumes and quality of training and production data. Data with appropriate authority and consent for use. Internal standards and functions, as well as ongoing maintenance and retraining requirements. Of course, this approach can also compete for talent in AI engineering.
Hybrid model It leverages existing public LLM and adapts to your own data using fine tuning and/or rigorous data filtering tools and techniques, providing some degree of customization. They can reduce positive investments and allow them to acquire gradually their internal capabilities and organizational skills.
Here we consider investing in technology and unique data preparation to protect personal, confidential or transactional information, create appropriate technical infrastructure with strict access controls and authentication, and provide ample documentation. Select a base model with a tweaking process and continue to evaluate the behavior, drift and use of the model to maintain detailed records for compliance.
Where should I start from?
To help you think through this, here are some important areas to start a board discussion:
Are you a regulated or highly sensitive business? Custom models keep data within the boundaries of the company, leak sensitive data, improve compliance controls, and adopt the latest solutions.
Do you want competitive, differentiated features? In that case, training your model on a unique, unique dataset will capture domain knowledge and create solutions tailored to your specific business and industry requirements.
Is a hybrid approach feasible? Using proprietary data carefully prepared in private cloud instances or air-gapped systems to fine-tune your public model might be a good solution. Ultimately, most public LLM users invest in RAG and other unique techniques to improve reliability, accuracy, and context specificity.
What problems are you trying to solve? Which approach is used will turn on the USECase context.
What can you afford? It is a context-specific location where to place your upfront investments (for example, towards developing reliable datasets, model development, or product development).
Finally, I would like to realistically evaluate:
Access to engineering talent, applicable and usable datasets, and capital n are necessary for competitive differentiation
Timing Requirements n Infrastructure Availability Generation AI raises build or purchase decisions, as is the case with existing Tech Stacks. For example, when interoperability is important, the utility of a universal solution can outweigh the risks associated with it. However, when competitive differentiation is important, universal LLM generates content that others can replicate (trained with all the same data). So it's possible that a unique approach is better. And, of course, if you can afford it.
