so slow

[cozmo14047]

Why is it so slow

[FotieMConstant]

Hey @cozmo14047 could you please provide more information on what exactly is slow?

for context, the project is still under development and it’s normal if you encounter some bugs etc. currently the tts makes a request online and voice recognition isn’t the best, yet.

please bear with me as I keep improving inference and the overall model.

also, it’s advisable to run it on a 16gb ram minimum and tests haven’t been made yet for other OSs other then MacOS 14.4.x

[cozmo14047]

hey sorry there was ment to be way more text i accedenillt clicked enter, so 2 of us have tried to run it and its really slow, both on high powered pcs. it takes about 4 min to respond and it responds 1 word every minuite. even a siple what time is it.

[FotieMConstant]

@cozmo14047 thanks for the heads up on this, could you please provide more content on the OS, system and version, RAM etc?

does ollama works well on those pcs? And are you able to run Jarvis in the command line with the command as seen here: https://ollama.com/fotiecodes/jarvis ?

please let me know, also the more content I have the better I will be able to help troubleshoot any issue you might have:)

[cozmo14047]

@cozmo14047 thanks for the heads up on this, could you please provide more content on the OS, system and version, RAM etc?

does ollama works well on those pcs? And are you able to run Jarvis in the command line with the command as seen here: https://ollama.com/fotiecodes/jarvis ?

please let me know, also the more content I have the better I will be able to help troubleshoot any issue you might have:)

hi thank you, windows 11 and 10 64 gb ram ollama works well and code works fine its just super slow

[cozmo14047]

for what we wanted to use it for response time needs to be under 3 seconds and im guessing that isint possbile

[FotieMConstant]

@cozmo14047 thanks for the heads up on this, could you please provide more content on the OS, system and version, RAM etc?
does ollama works well on those pcs? And are you able to run Jarvis in the command line with the command as seen here: https://ollama.com/fotiecodes/jarvis ?
please let me know, also the more content I have the better I will be able to help troubleshoot any issue you might have:)

hi thank you, windows 11 and 10 64 gb ram ollama works well and code works fine its just super slow

that's rather unexpected, although it has not been tested on windows if ollama works fine, you shouldn't have any issues.

[FotieMConstant]

for what we wanted to use it for response time needs to be under 3 seconds and im guessing that isint possbile

thanks for the feedback on this, it works super good on MacOS 14.4.x apple silicon with 16GB ram, 8 core GPU with a relatively better response time. the goal is to improve this of course as much as i can, so please keep an eye for updates. i might need to have someone help with testing on windows and linux as well.

[cozmo14047]

Well what do you mean works fine it doesnt throw errors and been tried on 2 diffrwnt pcs in 2 diffrent countrys

[FotieMConstant]

Well what do you mean works fine it doesnt throw errors and been tried on 2 diffrwnt pcs in 2 diffrent countrys

i am referring to ollama here, jarvis uses ollama which runs the llm locally after being downloaded from the hub

[FotieMConstant]

@cozmo14047 thanks for the heads up on this, could you please provide more content on the OS, system and version, RAM etc?
does ollama works well on those pcs? And are you able to run Jarvis in the command line with the command as seen here: https://ollama.com/fotiecodes/jarvis ?
please let me know, also the more content I have the better I will be able to help troubleshoot any issue you might have:)

hi thank you, windows 11 and 10 64 gb ram ollama works well and code works fine its just super slow

also, i will have to reproduce this error to better understand what we have here. Unfortunately i do not possess a windows computer with those specs.

[cozmo14047]

Well what do you mean works fine it doesnt throw errors and been tried on 2 diffrwnt pcs in 2 diffrent countrys

i am referring to ollama here, jarvis uses ollama which runs the llm locally after being downloaded from the hub

Yeah its not throwing any errors so seems to be fine

[FotieMConstant]

Well what do you mean works fine it doesnt throw errors and been tried on 2 diffrwnt pcs in 2 diffrent countrys

i am referring to ollama here, jarvis uses ollama which runs the llm locally after being downloaded from the hub

Yeah its not throwing any errors so seems to be fine

and the speed is good? i mean when using ollama? also are you able to chat with jarvis from the command line when you run the command

ollama run fotiecodes/jarvis

[cozmo14047]

Dint know how to run ollama alone but using that command i can chat its just really slow

[FotieMConstant]

Dint know how to run ollama alone but using that command i can chat its just really slow

Hi, thanks for this feedback, i think the issue is with you ollama itself, if jarvis is slow when you chat with it from the command line using the above command, you computer probably isn't capable to run llms. this is not a particular issue related to this project.

which is rather strange as you have a 64gb ram. what GPU you using please? NVIDIA's RTX?

[FotieMConstant]

@cozmo14047 are you able to run other llms with ollama? what is their various speeds?

[cozmo14047]

Dint know how to run ollama alone but using that command i can chat its just really slow

Hi, thanks for this feedback, i think the issue is with you ollama itself, if jarvis is slow when you chat with it from the command line using the above command, you computer probably isn't capable to run llms. this is not a particular issue related to this project.

which is rather strange as you have a 64gb ram. what GPU you using please? NVIDIA's RTX?

No idea what gpu but both pcs are less the a year old and ver powerful

[cozmo14047]

@cozmo14047 are you able to run other llms with ollama? what is their various speeds?

im not sure any commands that i can use to run other llms?

[cozmo14047]

ok im trying another model now

[cozmo14047]

ok so mine doesent like running other ones either so ill try again tommorow and ask my freind to test it to, seems like it could indeed be ollamam ill let you know tommorow if it is or not

[FotieMConstant]

If you are having any specific issues or concerns related to ollama, feel free to open an issue in their repo: https://github.com/ollama/ollama

in the main time, please keep me posted;)

[brainiakk]

@cozmo14047 I think it's more about your PC's CPU/Processor or GPU than the RAM, but the RAM is important too. I'm using a 16GB Macbook Pro M1 processor and it's pretty fast with ollama and even LMstudio API

[FotieMConstant]

@cozmo14047 I think it's more about your PC's CPU/Processor or GPU than the RAM, but the RAM is important too. I'm using a 16GB Macbook Pro M1 processor and it's pretty fast with ollama and even LMstudio API

I agree with you

[cozmo14047]

@cozmo14047 I think it's more about your PC's CPU/Processor or GPU than the RAM, but the RAM is important too. I'm using a 16GB Macbook Pro M1 processor and it's pretty fast with ollama and even LMstudio API

Hmm well our pcs are top line ill find out tommorow what processior i have, this isint going to be good for what i needed it for anyway, so thanks

[cozmo14047]

@cozmo14047 I think it's more about your PC's CPU/Processor or GPU than the RAM, but the RAM is important too. I'm using a 16GB Macbook Pro M1 processor and it's pretty fast with ollama and even LMstudio API

I agree with you

@FotieMConstant so the othe ollema ones work fine apart from the 40 gb one

[FotieMConstant]

@cozmo14047 I think it's more about your PC's CPU/Processor or GPU than the RAM, but the RAM is important too. I'm using a 16GB Macbook Pro M1 processor and it's pretty fast with ollama and even LMstudio API

I agree with you

@FotieMConstant so the othe ollema ones work fine apart from the 40 gb one

hi @cozmo14047, sorry I didn't quite catch that. you mean the models?

[cozmo14047]

@cozmo14047 I think it's more about your PC's CPU/Processor or GPU than the RAM, but the RAM is important too. I'm using a 16GB Macbook Pro M1 processor and it's pretty fast with ollama and even LMstudio API

I agree with you

@FotieMConstant so the othe ollema ones work fine apart from the 40 gb one

hi @cozmo14047, sorry I didn't quite catch that. you mean the models?

Yes the other models all of theme work f8ne and fast except for the big 40 gb one

[FotieMConstant]

Thank for this feedback, will look into it, perhaps work with a much lightweight model? we'll see. will keep you posted