Jul 23, 2023
Hi James, it will involve the use of the `gpu_layers` parameter e.g., llm = CTransformers(model='/path/to/ggml-llama-2.bin', model_type='llama', gpu_layers=50). You can find the info here: https://github.com/marella/ctransformers#gpu
Hi James, it will involve the use of the `gpu_layers` parameter e.g., llm = CTransformers(model='/path/to/ggml-llama-2.bin', model_type='llama', gpu_layers=50). You can find the info here: https://github.com/marella/ctransformers#gpu
Senior Data Scientist at Boston Consulting Group | Top Tech Author | 2M+ reads on Medium | linkedin.com/in/kennethleungty | github.com/kennethleungty