Team of 5 from Princeton, and we got funding to build a local inference engine for Apple Silicon - rust, hand written kernels - and we're at the point where working with ~100 people will expose bugs/what people want tool-wise. All of this is free open source - will remain so.
We're ahead of llama/mlx for small models working on similar performance for larger in the long run. Where this is going: the engine we're building supports a fully local agent that can do real work on your own files, apps, has permissions with OS kernel enforcement.
Asking for any feedback and if you're really interested we're opening up a waitlist and taking 100 people into free beta and working with them 1-on-1 to writing specific tools and performance engineering on setups (sign up at https://conifer.build/feedback). Please only do this if you imagine using this and have some idea in mind, we'll release a full version later this summer but we want to build around talent. We need real usage and unrestrained feedback from ppl who run local models.
site is live at conifer.build. also drop anything you want to see or ideas. conifer.build/feedback if you want to drop comment anon
[link] [comments]
