How To Build Your Own Custom ChatGPT With Custom Knowledge Base

Latte And Now For That Latte
1

This could be very useful for researchers. Could be like DEVONthink on steroids. It could also leak all your data.

But I would assume that the way things are going (at light speed) that soon there will be ways to have DIY AI and keep your data private.

I can just see Apple coming out with products that have the computing capacity (a GPT chip) to do the modeling and even a built in AI Chat program that deals with your data. That would certainly be a major selling point when buying a new Mac that other computers currently could not match.

2

M-Series CPUs come with 16 “Neural Engine” Cores (32 on M1 Ultra). That’s Apples fancy term for NPUs. And according to Apple, Core ML models “run strictly on the user’s device and remove any need for a network connection”. So, the ingredients are already there, both hardware and APIs.

1 Like
3

Are you sure it is robust enough? I don’t know because I haven’t upgraded to M chip models yet. When is the next update from Apple due?

4

In what way? I doesn’t rattle and no smoke comes out of it. Likely because those cores are bored to death.

Hardware or software? The first M3 series CPUs could be introduced later this year. The next major macOS update will be released this fall (as every year), with the usual “preview” WWDC keynote (June 5).

Wouldn’t be too surprised if they talk about a new supercharged Siri.

5

Then there is this. I can feed it my own data.

and

6

These models are computationally pretty simple. They’re really just lots of matrix multiplications. Any computer built in the last decade or more can run the model, and any computer with a GPU accelerator can run it decently well. Hobbyists run these things on Raspberry Pis.

Your hardware resources will limit how many parameters you can give the model and still have it run in a reasonable amount of time, and therefore how accurate a result you can get. (Or how complex a question you can ask, depending on how you look at it.)

Products like GPT4All use a data center to pre-train the model, so all the local hardware has to do is apply the existing weights to user-supplied data. That’s computationally pretty easy, but how effective it is will depend on how closely the training data resembles the user data. I wouldn’t expect a general purpose training set to give good results if applied to protein folding, say.